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Abstract 

We study a generalized Polya urn model with two types of ball. If the drawn ball is red it is 
replaced together with a black ball, but if the drawn ball is black it is replaced and a red ball 
is thrown out of the urn. When only black balls remain, the roles of the colours are swapped 
and the process restarts. We prove that the resulting Markov chain is transient but that if 
we throw out a ball every time the colours swap, the process is recurrent. We show that the 
embedded process obtained by observing the number of balls in the urn at the swapping times 
has a scaling limit that is essentially the square of a Bessel diffusion. We consider an oriented 
percolation model naturally associated with the urn process, and obtain detailed information 
about its structure, showing that the open subgraph is an infinite tree with a single end. We also 
study a natural continuous-time embedding of the urn process that demonstrates the relation to 
the simple harmonic oscillator; in this setting our transience result addresses an open problem 
in the recurrence theory of two-dimensional linear birth and death processes due to Kesten and 
Hutton. We obtain results on the area swept out by the process. We make use of connections 
between the urn process and birth-death processes, a uniform renewal process, the Eulerian 
numbers, and Lampcrti's problem on processes with asymptotically small drifts; we prove some 
new results on some of these classical objects that may be of independent interest. For instance, 
we give sharp new asymptotics for the first two moments of the counting function of the uniform 
renewal process. Finally we discuss some related models of independent interest, including a 
'Poisson earthquakes' Markov chain on the homeomorphisms of the plane. 

Keywords: Urn model, recurrence classification, oriented percolation, uniform renewal process, two- 
dimensional linear birth and death process, Bessel process, coupling, Eulerian numbers. 
AMS subject classification: Primary: 60J10; Secondary: 60J25, 60K05, 60K35. 

1 Introduction 

Urn models have a venerable history in probability theory, with classical contributions having 
been made by the Bernoullis and Laplace, among others. The modern view of many urn models 
is as prototypical reinforced stochastic processes. Classical urn schemes were often employed as 
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'thought experiments' in which to frame statistical questions; as stochastic processes, urn models 
have wide-ranging applications in economics, the physical sciences, and statistics. There is a large 
literature on urn models and their applications — see for example the monographs [20\ [30] and the 
surveys [251 [34] — and some important contributions have been made in the last few years: see 

eg. MUM- 

A generalized Polya urn with 2 types of ball, or 2 colours, is a discrete-time Markov chain 
(X n ,Y n ) n £z on Z+, where Z + := {0, 1,2,.. .}. The possible transitions of the chain are specified 
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This process can be viewed as an urn which at time n contains X n red balls and Y n black balls. At 
each stage, a ball is drawn from the urn at random, and then returned together with an red balls 
and Oj2 black balls, where i = 1 if the chosen ball is red and i = 2 if it is black. 

A fundamental problem is to study the long-term behaviour of (X n , Y n ), defined by ([I]), or some 
function thereof, such as the fraction of red balls X n /(X n + Y n ). In many cases, coarse asymptotics 
for such quantities are governed by the eigenvalues of the reinforcement matrix A (see e.g. [4j or 
[5j §V.9]). However there are some interesting special cases (see e.g. |35j). and analysis of finer 
behaviour is in several cases still an open problem. 



A large body of asymptotic theory is known under various conditions on A and its eigenvalues. 

I " 
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Often it is assumed that all a^- > 0; e.g., A = 

with a, b > specifies a Friedman urn. 



specifies the standard Polya urn, while A 



In general, the entries a^ may be negative, meaning that balls can be thrown away as well as 
added, but nevertheless in the literature tenability is usually imposed. This is the condition that 
regardless of the stochastic path taken by the process, it is never required to remove a ball of a 
colour not currently present in the urn. For example the Ehrenfest urn, which models the diffusion 

of a gas between two chambers of a box, is tenable despite its reinforcement matrix 

having some negative entries. 
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Departing from tenability, the OK Corral model is the 2-colour urn with reinforcement matrix 

^ . This model for destructive competition was studied by Williams and Mcllroy [33] and 

Kingman [23] (and earlier as a stochastic version of Lanchester's combat model; see e.g. [42] and 
references therein). Kingman and Volkov [25J showed that the OK Corral model can be viewed as 
a time-reversed Friedman urn with a = and 6 = 1. 

In this paper, we will study the 2-colour urn model with reinforcement matrix 



,4 



1 
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(2) 



To reiterate the urn model, at each time period we draw a ball at random from the urn; if it is 
red, we replace it and add an additional black ball, if it is black we replace it and throw out a 
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Figure 1: Two sample trajectories of the simple harmonic urn process, starting at (50,0) and 
running for about 600 steps (left) and starting at (1000,0) and running for 100,000 steps (right). 

red ball. The eigenvalues of A are ±i, corresponding to the ordinary differential equation v = Av, 
which governs the phase diagram of the simple harmonic oscillator. This explains the name simple 
harmonic urn. Naively, one might hope that the behaviour of the Markov chain is closely related 
to the paths in the phase diagram. We will see that it is, but that the exact behaviour is somewhat 
more subtle. 



2 Exact formulation of the model and main results 
2.1 The simple harmonic urn process 

The definition of the process given by the transition probabilities (pQ) and the matrix ([2]) only 
makes sense for X n ,Y n > 0; however, it is easy to see that almost surely (a.s.) X n < eventually. 
Therefore, we reformulate the process (X n ,Y n ) rigorously as follows. 

For zq € N := {1, 2, . . .} take (Xq,Yq) = (zo,0); we start on the positive x-axis for convenience 
but the choice of initial state does not affect any of our asymptotic results. For n € Z+, given 
(X n ,Y n ) = (x,y) 6Z 2 \ {(0,0)}, we define the transition law of the process by 



(X n -i-i, Y n+ \) 



(x, y + sgn(x)), with probability jj^, 

(3) 

(x - sgn(y), y), with probability jj^, 



where sgn(x) = —1,0,1 if x < 0, x = 0, x > respectively. The process (X n ,Y n ) n£ z + is an 
irreducible Markov chain with state-space Z 2 \ {(0,0)}. 

Let vq := 0, and recursively define stopping times 

u k := min{?7, > v k _i : X n Y n = 0}, (k £ N), 

where throughout the paper we adopt the usual convention min0 := oo. Thus (vk)keN is the 
sequence of times at which the process visits one of the axes. 
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It is easy to see that every v% is almost surely finite. Moreover, by construction, the process 
(X Vk ,Y Vk )k&i visits in cyclic (anticlockwise) order the half-lines {y > 0}, {x < 0}, {y < 0}, {x > 0}. 
It is natural (and fruitful) to consider the embedded process (Zk)kez + obtained by taking Zq := zq 
and Z k := \X Vk \ + \Y Vh \ (fc € N). 

If (X n ,Y n ) is viewed as a random walk on Z 2 , the process Zk is the embedded process of the 
distances from at the instances of hitting the axes. To interpret the process (X n ,Y n ) as the urn 
model described in Section [1] we need a slight modification to the description there. Starting with 
zq red balls, we run the process as described in Section [IJ so the process traverses the first quadrant 
via an up/left path until the red balls run out (i.e. we first hit the half-line {y > 0}). Now we 
interchange the roles of the red and black balls, and we still use y to count the black balls, but 
we switch to using — x to count the number of red balls. Now the process traverses the second 
quadrant via a left/down path until the black balls run out, and so on. In the urn model, Zk is 
the number of balls remaining in the urn when the urn becomes monochromatic for the k-th. time 
(k G N). 

The strong Markov property and the transition law of (X n ,Y n ) imply that Zk is an irreducible 
Markov chain on N. Since our two Markov chains just described are irreducible, there is the 
usual recurrence/transience dichotomy, in that either the process is recurrent, meaning that with 
probability 1 it returns infinitely often to any finite subset of the state space, or it is transient, 
meaning that with probability 1 it eventually escapes to infinity. Our main question is whether the 
process Zk is recurrent or transient. It is easy to see that, by the nature of the embedding, this 
also determines whether the urn model (X n ,Y n ) is recurrent or transient. 

Theorem 2.1. The process Zk is transient; hence so is the process (X n ,Y n ). 

Exploiting a connection between the increments of the process Zk and a renewal process whose 
inter-arrival times are uniform on (0, 1) will enable us to prove the following basic result. 

Theorem 2.2. Let n G N. Then 

E [Z k+ i | Z k = n] = n + ^ + O (e ai «) , (4) 

as n -> cc, where a x + foi = -(2.088843 . . . ) + (7.461489 . . . )i is a root of A - 1 + e" A = 0. 

The error term in @ is sharp, and we obtain it from new (sharp) asymptotics for the uniform 
renewal process: see Lemma 14.51 and Corollary 16.51 which improve on known results. To prove 
Theorem 12.11 we need more than Theorem 12.21 we need to know about the second moments of 
the increments of Zk, amongst other things; see Section [6j In fact we prove Theorem 12.11 using 
martingale arguments applied to h(Zk) for a well-chosen function h; the analysis of the function 
h(Zk) rests on a recurrence relation satisfied by the transition probabilities of Zk, which are related 
to the Eulerian numbers (see Section [3|) . 

2.2 The leaky simple harmonic urn 

In fact the transience demonstrated in Theorem 12.11 is rather delicate, as one can see by simulating 
the process. To illustrate this, we consider a slight modification of the process, which we call 
the leaky simple harmonic urn. Suppose that each time the roles of the colours are reversed, the 
addition of the next ball of the new colour causes one ball of the other colour to leak out of the 
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urn; subsequently the usual simple harmonic urn transition law applies. If the total number of balls 
in the urn ever falls to one, then this modified rule causes the urn to become monochromatic at 
the next step, and again it contains only one ball. Thus there will only be one ball in total at all 
subsequent times, although it will alternate in colour. We will see that the system almost surely 
does reach this steady state, and we obtain almost sharp tail bounds on the time that it takes to do 
so. The leaky simple harmonic urn arises naturally in the context of a percolation model associated 
to the simple harmonic urn process, defined in Section [2.41 below. 

As we did for the simple harmonic urn, we will represent the leaky urn by a Markov chain 
(X' n , Y^). For this version of the model, it turns out to be more convenient to start just above the 
axis; we take (X' ,Yq) = (zq, 1), where zq € N. The distribution of {X' n+l ,Y^ +l ) depends only on 
(X' n ,Y£) = (x,y). 

If xy 7^ 0, the transition law is the same as that of the simple harmonic urn process. The 
difference is when x = or y = 0; then the transition law is 

(K+i, Y n+i) = (- sgn(y), y - sgn(y)) (x = 0) 
= {x- sgn(x),sgn(x)) (y = 0). 

Now (X' n ,Y^) is a reducible Markov chain whose state-space has two communicating classes, the 
closed class C = {(x, y) G 1? : \x\ + \y\ = 1} and the class {(x, y) € 1? : \x\ + \y\ > 2}; if the process 
enters the closed class C it remains there for ever, cycling round the origin. Let r be the hitting 
time of the set C, that is 

r :=mf{n eZ: \X' n \ + \Y^\ = 1}. 

Theorem 2.3. For the leaky urn, P(r < oo) = 1. Moreover, for any e > 0, E[r 1_e ] < oo but 
E[t 1+£ ] = oo. 

In contrast, Theorem 12.11 implies that the analogue of r for the ordinary urn process has P(r = 
oo) > if z > 2. 

2.3 The noisy simple harmonic urn 

In view of Theorems 12.11 and 12 . 3} it is natural to ask about the properties of the hitting time r if at 
the time when the balls of one colour run out we only discard a ball of the other colour with some 
probability p G (0, 1). For which p is r a.s. finite? (Answer: for p > 1/3; see Corollary 12.71 below.) 

We consider the following natural generalization of the model specified by ([3]) in order to probe 
more precisely the recurrence/transience transition. We call this generalization the noisy simple 
harmonic urn process. In a sense that we will describe, this model includes the leaky urn and also 
the intermittent leaky urn mentioned at the start of this section. The basic idea is to throw out (or 
add) a random number of balls at each time we are at an axis, generalizing the idea of the leaky 
urn. It is more convenient here to work with irreducible Markov chains, so we introduce a 'barrier' 
for our process. We now describe the model precisely. 

Let «, «i, «2: ■ ■ ■ be a sequence of i.i.d. Z- valued random variables such that 

E[e A|K| ] < oo, (5) 

for some A > 0, so in particular E[k] is finite. We now define the Markov chain {X n ,Y n ) n( zj i+ for 
the noisy urn process. As for the leaky urn, we start one step above the axis: let zq G N, and take 
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(Xq,Yo) = (zq, 1). For n £ Z+, given (X n , Y n ) = (x, y) £ Z 2 \ {(0, 0)}, we define the transition law 
as follows. If xy ^ then 



x 



y + sgn(x)), with probability jj^, 
(x — sgn(y), y), with probability ■ 



\x\+\vV 



while if x = or y = we have 



(X n+1 ,Y n+ i) = (-sgn(y), sgn(y)max(l, \y\ - K n )) (x = 0), 
(X n+1 ,Y n+1 ) = (sgn(x)max(l, \x\ - K n ), sgn(x)) (y = 0). 

In other words, the transition law is the same as ([3]) except when the process is on an axis at time 
n, in which case instead of just moving one step away in the anticlockwise perpendicular direction 
it also moves an additional distance K n parallel to the axis towards the origin (stopping distance 1 
away if it would otherwise reach the next axis or overshoot). Then (X n ,Y n ) n ^ + is an irreducible 
Markov chain on Z 2 \ {(0,0)}. The case where P(k = 0) = 1 corresponds to the original process 
(X n ,Y n ) starting one unit later in time. 

A fundamental random variable is the first passage time to within distance 1 of the origin: 

r := min{n E Z + : \X n \ + \Y n \ = 1} = min{n £ Z + : (X n , Y n ) £ C}. 

Define a sequence of stopping times by setting Dq := — 1 and for k £ N, 

v k := min{n > i> k _ x : X n Y n = 0}. 

As an analogue of Z k , set Zq := zq and for k £ N define 

Z k := max{\X 1+i>k \, \Y 1+C/k \} = \X 1+C/k \ + |Yi+,>J - 1; 

then (^fc)fcgz+ is an irreducible Markov chain on N. Define the return-time to the state 1 by 

r q := min{fc £ N : Z k = 1}, (6) 

where the subscript q signifies the fact that a time unit is one traversal of a quadrant here. By our 
embedding, r = v T . 

Note that in the case F(k = 0) = 1, (Z k ) ke % + has the same distribution as the original {Z k ) k€ z + - 
The noisy urn with P(k = 1) = 1 coincides with the leaky urn described in Section [2.21 up until the 
time r (at which point the leaky urn becomes trapped in C). Similarly the embedded process Z k 
with P(k = 1) = 1 coincides with the process of distances from the origin of the leaky urn at the 
times that it visits the axes, up until time T q (at which point the leaky urn remains at distance 1 
forever). Thus in the P(k = 1) = 1 cases of all the results that follow in this section, r and r q can 
be taken to be defined in terms of the leaky urn (X' n , Y^). 

The next result thus includes Theorem 1 2 . 1 1 and the first part of Theorem 12.31 as special cases. 
Theorem 2.4. Suppose that k satisfies ([5^. Then the process Z k is 

(i) transient i/E[/c] < 1/3; 



6 



(ii) null- recurrent if 1/3 < E[/c] < 2/3; 
(mj positive- recurrent «/E[ac] > 2/3. 

Of course, part (i) means that P(t 9 < oo) < 1, part (ii) that P(t 9 < oo) = 1 but E[r 9 ] = oo, and 
part (iii) that E[t 9 ] < oo. We can in fact obtain more information about the tails of T q : 

Theorem 2.5. Suppose that k satisfies ^) and E[k] > 1/3. Then E["7g] < oo /or p < 3E[k] — 1 
and E[t£] = oo for p > 3E[«] - 1. 

It should be possible, with some extra work, to show that E[rg] = oo when p = 3E[k] — 1, using 
the sharper results of [2] in place of those from [3] that we use below in the proof of Theorem 12.51 

In the recurrent case, it is of interest to obtain more detailed results on the tail of r (note that 
there is a change of time between r and r q ). We obtain the following upper and lower bounds, 
which are close to sharp. 

Theorem 2.6. Suppose that k satisfies (0j and E[k] > 1/3. Then E[t p ] < oo for p < — \ an d 

E[t'P] = oo for p> S^hi. 

Theorems 12.41 and 12.61 have an immediate corollary for the noisy urn process (X n ,Y n ). 

Corollary 2.7. Suppose that k satisfies ([5|). The noisy simple harmonic urn process (X n ,Y n ) is 
recurrent if E[/c] > 1/3 and transient if E[«] < 1/3. Moreover, the process is null-recurrent if 
1/3 < E[k] < 1 and positive-recurrent z/E[k] > 1. 

This result is close to sharp but leaves open the question of whether the process is null- or 
positive-recurrent when E[«] = 1 (we suspect the former). 

We also study the distributional limiting behaviour of Z^ in the appropriate scaling regime 
when E[k] < 2/3. Again the case P(/c = 0) = 1 reduces to the original Zj-. 

Theorem 2.8. Suppose that k satisfies (GP and that E[k] < 2/3. Let (-Dt)tg[o,i] be a diffusion 
process taking values in IR + := [0, oo) with Dq = and infinitesimal mean fi(x) and variance o~ 2 (x) 
given for x G M+ by 

2 2 

K X ) = 3 ~ E M ; ct2 (^) = g^- 

T/ien as k ^ oo, 

(A; - ^) te [o,i] -> (A)t G [o,i], 

where the convergence is in the sense of finite- dimensional distributions. Up to multiplication by a 
scalar, Dt is the square of a Bessel process with parameter 4 — 6E[/c] > 0. 

Since a Bessel process with parameter 7 G N has the same law as the norm of a 7-dimensional 
Brownian motion, Theorem 12.81 says . for example, that if E[/c] = (e.g., for the original urn process) 
the scaling limit of Zt is a scalar multiple of the norm-square of 4-dimensional Brownian motion, 
while if E[k] = 1/2 the scaling limit is a scalar multiple of the square of one-dimensional Brownian 
motion. 

To finish this section, consider the area swept out by the path of the noisy simple harmonic urn 
on its first excursion (i.e., up to time r). Additional motivation for studying this random quantity 
is provided by the percolation model of Section 12.41 Formally, for n G N let T n be the area of the 
triangle with vertices (0,0), (X n _i, Yn-i), arid (X ni Y n ), and define A := Yln=i^n- 
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Theorem 2.9. Suppose that k satisfies 

(i) Suppose that E[k] < 1/3. Then F(A = oo) > 0. 

(ii) Suppose that E[k] > 1/3. Then E[AP] < oo for p < M£bl. 

In particular, part (ii) gives us information about the leaky urn model, which corresponds to 
the case where P(/c = 1) = 1, at least up until the hitting time of the closed cycle; we can still make 
sense of the area swept out by the leaky urn up to this hitting time. We then have E[A P ] < oo for 
p < 2/3, a result of significance for the percolation model of the next section. We suspect that the 
bounds in Theorem I2.9f ii) are tight. We do not prove this but have the following result in the case 
P(« = 1) = 1. 

Theorem 2.10. Suppose P(« = 1) = 1 (or equivalently take the leaky urn). Then EL4] = oo. 
2.4 A percolation model 

Associated to the simple harmonic urn is a percolation model which we describe in this section. 
The percolation model, as well as being of interest in its own right, couples many different instances 
of the simple harmonic urn, and exhibits naturally an instance of the leaky version of the urn in 
terms of the planar dual percolation model. Our results on the simple harmonic urn will enable us 
to establish some interesting properties of the percolation model. 

The simple harmonic urn can be viewed as a spatially inhomogeneous random walk on a directed 
graph whose vertices are Z 2 \ {(0, 0)}; we make this statement more precise shortly. In this section 
we will view the simple harmonic urn process not as a random path through a predetermined 
directed graph but as a deterministic path through a random directed graph. To do this it is 
helpful to consider a slightly larger state-space which keeps track of the number of times that the 
urn's path has wound around the origin. We construct this state-space as the vertex set of a graph 
G that is embedded in the Riemann surface 1Z of the complex logarithm, which is the universal 
cover of R 2 \ {(0,0)}. To construct G, we take the usual square-grid lattice and delete the vertex 
at the origin to obtain a graph on the vertex set Z 2 \ {(0, 0)}. Make this into a directed graph by 
orienting each edge in the direction of increasing argument; the paths of the simple harmonic urn 
only ever traverse edges in this direction. Leave undirected those edges along any of the co-ordinate 
axes; the paths of the simple harmonic urn never traverse these edges. Finally, we let G be the lift 
of this graph to the covering surface 1Z. 

We will interpret a path of the simple harmonic urn as the unique oriented path from some 
starting vertex through a random subgraph H of G. For each vertex v of G the graph H has 
precisely one of the out-edges from v that are in G. If the projection of v to Z 2 is (x,y), then the 
graph H contains the edge from v that projects onto the edge from (x,y) to (x — sgn(y),y) with 
probability |y|/(|x| + |y|), and otherwise it contains the edge from v that projects onto the edge 
from (x, y) to (x, y + sgn(x)). These choices are to be made independently for all vertices v of G. In 
particular H does not have any edges that project onto either of the co-ordinate axes. The random 
directed graph H is an oriented percolation model that encodes a coupling of many different paths 
of the simple harmonic urn. To make this precise, let vq be any vertex of G. Then there is a 
unique oriented path vo, v±, V2, ■ ■ ■ through H. That is, (vi,Vi + \) is an edge of H for each i > 0. 
Let the projection of from 1Z to M 2 be the point (Xi,Yi). Then the sequence (Xi,Yi)^. is a 
sample of the simple harmonic urn process. If wq is another vertex of G, with unique oriented path 
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Figure 2: Simulated realizations of the simple harmonic urn percolation model: on a single sheet 
of 1Z (left) and on a larger section (right). 



wo, w\,W2, • • • , then its projection to Z 2 is also a sample path of the simple harmonic urn process, 
but we will show (see Theorem 12.111 below) that with probability one the two paths eventually 
couple, which is to say that there exist random finite m > and n > such that for all i > 
we have Vi+ m = u>i+ n - Thus the percolation model encodes many coalescing copies of the simple 
harmonic urn process. Next we show that it also encodes many copies of the leaky urn of Section 
IP 

We construct another random graph H' that is the dual percolation model to H. We begin with 
the planar dual of the square-grid lattice, which is another square-grid lattice with vertices at the 
points (7n + l/2,n-|-l/2), 777,n € Z. We orient all the edges in the direction of decreasing argument, 
and lift to the covering surface 1Z to obtain the dual graph G' . Now let H' be the directed subgraph 
of G' that consists of all those edges of G' that do not cross an edge of H. It turns out that H' can 
be viewed as an oriented percolation model that encodes a coupling of many different paths of the 
leaky simple harmonic urn. 

To explain this, we define a mapping $ from the vertices of G' to Z 2 . Let (x,y) be the co- 
ordinates of the projection of v G G' to the shifted square lattice Z 2 + (1/2, 1/2). Then 

= (^x + ^sgny,-(y - ^sgnx)^ . 

Thus we project from 1Z to M 2 , move to the nearest lattice point in the clockwise direction, and 
then reflect in the x-axis. If vq is any vertex of H' , there is a unique oriented path v$,vx,V2, ■ ■ ■ 
through H', this time winding clockwise. Take vq = (zq — 1/2, 1/2). A little thought shows that the 
sequence (XLY?) = & (vi) has the distribution of the leaky simple harmonic urn process. This is 
because the choice of edge in H' from v is determined by the choice of edge in H from the nearest 
point of G in the clockwise direction. The map <I> is not quite a graph homomorphism onto the 
square lattice because of its behaviour at the axes; e.g., it sends (3|, Jj) and (3^, —5) to (4,0) and 
(3, 1) respectively. The decrease of 1 in the x-co-ordinate corresponds to the leaked ball in the 
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leaky urn model. If some Vi has projection (xi,yi) with \xi\ + \yi\ = 1, then the same is true of all 
subsequent vertices in the path. This corresponds to the closed class C. 

From results on our urn processes, we will deduce the following quite subtle properties of the 
percolation model H. Let I(v) denote the number of vertices in the in-graph of the vertex v in H, 
which is the subgraph of H induced by all vertices from which it is possible to reach v by following 
an oriented path. 

Theorem 2.11. Almost surely, the random oriented graph H is, ignoring orientations, an infinite 
tree with a single semi-infinite end in the out direction. In particular, for any v, I(v) < oo a.s. and 
moreover M[I(v ) p ] < oo for any p < 2/3; however, K[I(v)] = oo. 

The dual graph H' is also an infinite tree a.s., with a single semi-infinite end in the out direction. 
It has a doubly-infinite oriented path and the in-graph of any vertex not on this path is finite a.s. 

2.5 A continuous-time fast embedding of the simple harmonic urn 

There is a natural continuous-time embedding of the simple harmonic urn process. Let (A(t), B(t))t£M. + 
be a Z 2 -valued continuous-time Markov chain with ^4(0) = ao, B(0) = bo, and transition rates 



Given that (A(t),B(t)) = (a, b), the wait until the next jump after time t is an exponential random 
variable with mean l/(|a| + |6|). The next jump is a change in the first co-ordinate with probability 
|6|/(|a| + so the process considered at its sequence of jump times does indeed follow the law of 
the simple harmonic urn. Note that the process does not explode in finite time since the jump rate 
at (a, b) is \a\ + and \X n \ + \Y n \ = 0(n) (as jumps are of size 1), so X^nd^l + l-^l)" 1 = 00 a - s - 

The process (A(t),B(t)) is an example of a two-dimensional linear birth and death process. The 
recurrence classification of such processes defined on was studied by Kesten [22] and Hutton [T7] . 
Our case (which has B\ \ + I?2,2 = in their notation) was not covered by the results in [221 [T7] ; 
Hutton remarks [171 p. 638] that "we do not yet know whether this case is recurrent or transient." In 
the setting of [221 117]. the boundaries of the quadrant would become absorbing in our case. The 
model on Z 2 considered here thus seems a natural setting in which to pose the recurrence/transience 
question left open by [22} 117]. Our Theorem 12.11 implies that (A(t),B(t)) is in fact transient. 

We call (A(t), B(t)) the fast embedding of the urn since typically many jumps occur in unit time 
(the process jumps faster the farther away from the origin it is). There is another continuous-time 
embedding of the urn model that is also very useful in its analysis, the slow embedding described 
in Section [3] below. 

The mean of the process (A(t), B(t)) precisely follows the simple harmonic oscillation suggested 
by the name of the model. This fact is most neatly expressed in the complex plane C. Recall that 
a complex martingale is a complex-valued stochastic process whose real and imaginary parts are 
both martingales. 

Lemma 2.12. The process (Mt)teM.+ defined by 



F(A(t + dt) = A(t) - sgn(5(t))) 
F(B(t + dt) = B(t) + sgn(A(t))) 



\B(t)\dt, 
\A(t)\dt. 



M t := e~ lt (A(t) + iB{t)) 
is a complex martingale. In particular, for t > to and zeC, 

E[A(t) + iB(t) | A(t ) + %B(to) = z] 



= z& 



,i(*-*o) 
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As can be seen directly from the definition, the continuous-time Markov chain (A(t), B(t)) 
admits a constant invariant measure; this fact is closely related to the 'simple harmonic flea circus' 
that we describe in Section [10. li 

Returning to the dynamics of the process, what is the expected time taken to traverse a quadrant 
in the fast continuous-time embedding? Define tj := inf{t € M + : Ait) = 0}. We use the notation 
P n ( • ) for P( • | A(0) = n, B(0) = 0), and similarly for E n . Numerical calculations strongly suggest 
the following: 

Conjecture 2.13. Let n £ N. With a>i « —2.0888 as in Theorem 1 2. 2\ above, as n — > oo, 

E n [r f ] = 7r/2 + 0(e ai 7v^)- 



We present a possible approach to the resolution of Conjecture 12. 131 in Section [9.3( it turns out 
that E n [ry] can be expressed as a rational polynomial of degree n evaluated at e. The best result 
that we have been able to prove along the lines of Conjecture 12.131 is the following, which shows 
not only that E n [T/] is close to tt/2 but also that Tf itself is concentrated about n/2. 

Theorem 2.14. Let n £ N. For any 5 > 0, as n — > oo, 

E n [T f ]=ir/2 + 0(n s -^), (7) 
E n [|r / -(vr/2)| 2 ] = 0(n 5 "( 1 / 2 )). (8) 



In the continuous-time fast embedding the paths of the simple harmonic urn are a discrete 
stochastic approximation to continuous circular motion at angular velocity 1, with the radius of 
the motion growing approximately linearly in line with the transience of the process. Therefore a 
natural quantity to examine is the area enclosed by a path of the urn across the first quadrant, 
together with the two co-ordinate axes. For a typical path starting at (n, 0) we would expect this 
to be roughly 7rn 2 /4, this being the area enclosed by a quarter-circle of radius n about the origin. 
We use the percolation model to obtain an exact relation between the expected area enclosed and 
the expected time taken for the urn to traverse the first quadrant. 

Theorem 2.15. For ti£N, for any 5 > 0, 

n 2 

E n [Area enclosed by a single traversal] = mE m [rj] = — — h 0{n^^ 2 ^ +& ). 

m=l 



In view of the first equality in Theorem 12.151 and Conjecture 12.131 we suspect a sharp version 
of the asymptotic expression for the expected area enclosed to be 

E„[Area enclosed by a single traversal] = h c + O (\/ne ain ) , 

for some constant c£R. 



2.6 Outline of the paper and related literature 

The outline of the remainder the the paper is as follows. We begin with a study of the discrete- 
time embedded process in the original urn model. In Section we use a decoupling argument to 



11 



obtain an explicit formula, involving the Eulerian numbers, for the transition probabilities of In 
Section H] we study the drift of the process Z^ and prove Theorem 12.21 We make use of an attractive 
coupling with the renewal process based on the uniform distribution. Then in Section [5] we give 
a short, stand-alone proof of our basic result, Theorem 12.11 In Section [6] we study the increments 
of the process Z^, obtaining tail bounds and moment estimates. As a by-product of our results 
we obtain (in Lemma 14.51 and Corollary I6.5P sharp expressions for the first two moments of the 
counting function of the uniform renewal process, improving on existing results in the literature. 
In Section [7] we study the asymptotic behaviour of the noisy urn embedded process Zj., building on 
our results on Z^. Here we make use of powerful results of Lamperti and others on processes with 

~ 1 /2 

asymptotically zero drift, which we can apply to the process Z k . Then in Section [8] we complete 
the proofs of Theorems I2.3H2.61 12.81 12.91 and 12.111 In Section [9] we study the continuous-time fast 
embedding described in Section 12.51 and in Sections 19.11 and 19.21 present proofs of Theorems 12.101 
12.141 an d 12.151 In Section 19.31 we give some curious exact formulae for the expected area and time 
described in Section 12.51 Finally, in Section [10] we collect some results on several models that are 
not directly relevant to our main theorems but that demonstrate further some of the surprising 
richness of the phenomena associated with the simple harmonic urn and its generalizations. 

We finish this section with some brief remarks on modelling applications related to the simple 
harmonic urn. The simple harmonic urn model has some similarities to R.F. Green's urn model for 
cannibalism (see e.g. |36j). The cyclic nature of the model is similar to that of various stochastic or 
deterministic models of certain planar systems with feedback: see for instance [12] and references 
therein. Finally, one may view the simple harmonic urn as a gated polling model with two queues 
and a single server. The server serves one queue, while new arrivals are directed to the other 
queue. The service rate is proportional to the ratio of the numbers of customers in the two queues. 
Customers arrive at the unserved queue at times of a Poisson process of constant rate. Once the 
served queue becomes empty, the server switches to the other queue, and a new secondary queue 
is started. This model gives a third continuous-time embedding of the simple harmonic urn, which 
we do not study any further in this paper. This polling model differs from typical polling models 
studied in the literature (see e.g. [29]) in that the service rate depends upon the current state of 
the system. One possible interpretation of this unusual service rate could be that the customers in 
the primary queue are in fact served by the waiting customers in the secondary queue. 

3 Transition probabilities for Zk 

In this section we derive an exact formula for the transition probabilities of the Markov chain 
(Zk)kez + (see Lemma I3T31 below). We use a coupling (or rather 'decoupling') idea that is sometimes 
attributed to Samuel Karlin and Herman Rubin. This construction was used in [23] to study the OK 
Corral gunfight model, and is closely related to the embedding of a generic generalized Polya urn in 
a multi-type branching process [H [5] . The construction yields another continuous-time embedding 
of the urn process, which, by way of contrast to the embedding described in Section 12.51 we refer 
to as the slow embedding of the urn. 

We couple the segment of the urn process (X n ,Y n ) between times + 1 and vu+i with cer- 
tain birth and death processes, as follows. Let Afc := l/k. Consider two independent Z + -valued 
continuous-time Markov chains, U(t) and V(t), t £ R + , where U(t) is a pure death process with 
the transition rate 

F{U(t + dt) = U{t) - 1 | U(t) = a) = X a dt, 
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and V(t) is a pure birth process with 

P{V(t + dt) = V(t) + 1 | V(t) = b) = X h dt. 

Set U(0) = z and V(0) = 1. 

From the standard exponential holding-time characterization for continuous-time Markov chains 
and the properties of independent exponential random variables, it follows that the embedded 
process (U(t),V(t)) considered at the times when either of its coordinates changes has the same 
distribution as the simple harmonic urn (X n ,Y n ) described above when (X n ,Y n ) is traversing the 
first quadrant. More precisely, let 9q := and define the jump times of the process V(t) — U(t) for 
n G N: 

6 n := inf{t > fl n _i : U{t) < f7(0„_i) or V{t) > V(O n -i)}. 

Since A5 < 1 for all b, the processes U(i), V(t) a.s. do not explode in finite time, so 9 n — > 00 a.s. as 
n — > 00. Define rj := min{n G N : U(9 n ) = 0} and set 

T:=e v = wf{t > : U{t) = 0}, 

the extinction time of U(t). The coupling yields the following result (cf [51 §V.9.2]). 

Lemma 3.1. Let k G Z+ and z G N. The sequence (U{9 n ),V(0 n )), n = 0, 1, . . . , rj, with (U(0), V(0)) = 
(z,l), has the same distribution as each of the two sequences 

(i) (|-Xn|> \Yn\)> n = ffc + 1, . . . , Vk+i, conditioned on Zf- = z and Y Uk = 0; 

(ii) (\Y n \, \X n \), n = Vk + 1, . . . , Vk+i, conditioned on Z\~ = z and X Uk = 0. 

Note that we set V(0) = 1 since (X Uk+ i,Y Uk+ i) is always one step in the 'anticlockwise' lattice 
direction away from (X Uk , Y Vk ) . Let 

T4 := inf{t > : V{t) = w}. 

We can represent the times T and as sums of exponential random variables. Write 

T z = ^kt k , and T' w = Y,Kk, (9) 

k=l k=l 

where ^1, Ci, £,2, C2, ■ ■ ■ are independent exponential random variables with mean 1. Then setting 
T = Tfjr \, (J9]) gives useful representations of T and T' w . 

As an immediate illustration of the power of this embedding, observe that Z^+i < Z^, if and 
only if V has not reached U(0) + 1 by the time of the extinction of U, i.e. T^( )+i > ^ • -^ u ^ © 
shows that T' z+1 and T z are identically distributed continuous random variables, so: 

Lemma 3.2. For z G N, P (Z k+1 < Z k \ Z k = z) = P (z£ (0)+1 > T u{0 ^ = \. 

We now proceed to derive from the coupling described in Lemma 13.11 an exact formula for the 
transition probabilities of the Markov chain (^fc)fcez + • Define p(n, m) = P {Z k+ i = m \ Z k = n). It 
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turns out that p(n,m) may be expressed in terms of the Eulerian numbers A(n,k), which are the 
positive integers defined for n G N by 

A(n,k) = ^(-l)^ n+1 V-;)", fc€{l,. ..,„}. 

The Eulerian numbers have several combinatorial interpretations and have many interesting prop- 
erties; see for example Bona [HI Chapter 1]. 

Lemma 3.3. For n, m £ N, the transition probability p(n, m) is given by 

(m - r ) n+m ~ l 



p(n,m) = m 1 



r! in + m — r)\ 

m .. . . 

^4(ra + m — 1, n) . 



(m + n)! 

We give two proofs of Lemma 13.31 both using the coupling of Lemma 13.11 but in quite different 
ways. The first uses moment generating functions and is similar to calculations in [23], while 
the second involves a time-reversal of the death process and makes use of the recurrence relation 
satisfied by the Eulerian numbers. Each proof uses ideas that will be useful later on. 

First proof of Lemma 13.31 By Lemma 13.11 the conditional distribution of Z^+x on = n 
coincides with the distribution conditional of V(T) on U(0) = n. So 

F(Z k+1 >m \Z k = n)= F(V(T) > m | 17(0) = n) = P(T n > T^ +1 ), (10) 

using the representations in ([9|). Thus from ([9]) and (|10p . writing 



i=l j=l 

we have that ¥(Zk+i > m \ Z^ = n) = P(i2 n>OT > 0). The density of R n ^ m can be calculated using 
the moment generating function and partial fractions; for t > 0, 

ii 1 /*-* /ivi+* fi 1 -^ 



^ 1 - it ^ 1 

8=1 7 = 1 



-l+,r 

for some coefficients Oj = a%; n ,m and 6j = bj ;n>m . Multiplying both sides of the last displayed 
equality by ni=i(l ~~ ^) TYJLii^ + J*) an d setting i = 1/i we obtain 

a * = n i _ n i + (A;/i ) = i-ir-H^- 1 n — n 

3=1 U/ ' fc=l V 7 ' J=l J j=i+l J k = l 

Simplifying, and then proceeding similarly but taking t = to identify bj, we obtain 

(]\n—ijn+m (_-\\m—jjn+m 

[n — i)\(m + i)\ (m — j)'(n + j)\ 
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Consequently the density of Rn, m is 

f E?=i^i- l e- x/ \ ifx>0; 

r(x) = < 

X7=i^rv/ J , ifx<o. 

Thus we obtain 

n 

F(Z k+l >m\Z k = n) = F(Z k+1 >m + l\ Z k = n) = F(R, hm > 0) = a ^ 



fc=i 



Z-^ (n — h\\(m -I- fcM 



(n — k)\(m + A;)! ^— ' i\(m + n — i)\ 



(m + 77,)! ^ V i J 



(n — i) 



n+m 



It follows that 
p(n, to) 



¥(Z k+1 >m\Z k = n)- ¥(Z k+1 > m + 1 | Z k = n) 
-lY(n- i) n + m ~ l ^ (-iY( n -i) n+m 



E 

i=0 



i\(m — 1 + n — i)\ 



E 

i=0 



i\{m + n — i)l 



™ ( — lY( n — fin+rn-l 

/Z , -m — [(m + n-i) - (n - i) 



i=0 



i!(m + n — i)\ 



m 



to 



(m + nj!^^ \ i J (n + m)] 



A(m + n — 1, n) 



(11) 



□ 



as required. 

Second proof of Lemma 13.31 Consider the birth process W(t) defined by VF(0) = 1 and 

W(t) = min{z £ Z+ : T z > t}, (t > 0), 

where T z is defined as in (0). The inter-arrival times of W(t) are (i^)| =1 and, given U(0) = z, the 
death process U(t) has the same inter-arrival times but taken in the reverse order. The processes 
V{t) and W(t) are independent and identically distributed. Define for n, m S N, 

r(n, to) = P (3 f > : W(t) = n, V(t) = m | V(0) = W(0) = 1) . 

If Z k = n, then Z k+ \ is the value of V when W first reaches the value n + 1; -Zfc+i = to if and 
only if the process (W, V) reaches (n, to) and then makes the transition to (n + 1, to). Since (W, V) 
is Markov, this occurs with probability r(n,m) n ™ m . So for n, m € N, 



p(n, to) 



to 



r(n, to) 



n + to 

Conditioning on the site from which (W, V) jumps to (n, to), we get, for n, m G N, n + to > 3, 

to , n 



(12) 



r(n, to) 



n + m 



— r(n — 1, to) + 



n + to — 1 



r(n, to — 1), 



(13) 
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where r(0,m) = r(n, 0) = 0. It is easy to check that r(fc, 1) = r(l, k) = l/k\ for all k G N. It will 
be helpful to define 

s(n,m) = [n + to — 1)! r(n, to) . 
Then we have for n, to G N, n + m > 3, 

s(n,m) = to s(n — 1,to) + n s(n,m — 1) , 

1) = s(l, fc) = 1 for all ken. 

These constraints completely determine the positive integers s(n,m) for all to, n G N. Since the 
Eulerian numbers ^4(n + m — 1,to) satisfy the same initial conditions and recurrence relation [6j 
Thm. 1.7], we have s(n,m) = A(m + n — 1, to), which together with (I12|) gives the desired formula 
for p(n,m). □ 

It is evident from (1131) and its initial conditions that r(n, to) = r(m,n) for all n, to € N. So 

np(n,m) = mp(m,n) , (14) 

Therefore the a-finite measure 7r on N defined by 7r(n) = n satisfies the detailed balance equations 
and hence is invariant for p(-, •). In fact there is a pathwise relation of the same type, which we 
now describe. We call a sequence ui = (xj,yj)j =0 (k > 2) of points in an admissible traversal 
if yo = x k = 0, x > 1, y fe > 1, each point (xj,yj), 2 < j < k - 1 is one of (xj-i - 1, t/j-i), 
(xj-i,yj-i + 1), and (x 1( j/i) = (x ,yo + 1), {xk,Vk) = (xk-i ~ hVk-l)- If ^ is an admissible 
traversal, then so is the time-reversed and reflected path uj' = (yk-j, %k-j)j=o- I n f ac *' conditioning 
on the endpoints, uj and uj' have the same probability of being realized by the simple harmonic urn: 

Lemma 3.4. For any admissible traversal (xj, yj)j =0 with xq = n G N, yk = to G N, 
P((X,-,1S)£=o = ( o | Z = n,Zi =to) = P((X J ,y j )JLo = a*-i)J=o I Z = m,Z 1 = n). 

Proof. Let cj = {xj,yj)j =0 be an admissible traversal, and define 

P = P(w) = P((Xj, l^)iio = 0^' %')i=o> Z\ = m \ Zq = n), and 

p' = p '(a;) = P((Xj, 15)^o = (j/fc-i, a:fc-j)J=o, ^1 = ™ I ^0 = to), 
so that p' is the probability of the reflected and time-reversed path. To prove the lemma it suffices 
to show that for any uj with (xo,yo) = (n,0) and (xk,yk) = (0, to), p(uj)/p(n,m) = p'(uj)/p(m,n). 
In light of (]14h . it therefore suffices to show that np = mp'. To see this, we use the Markov property 
along the path uj to obtain 

fe-i 

P = + !J ' ] '{.<,.: •',} + Z// 1 ^,., 

j=0 

while, using the Markov property along the reflection and reversal of uj, 
fe-l 

P' = YliXk-j + yk-j)~ 1 (x k -jl {a . k _._ 1=mk _. } + Vh-j^-i-X^lh-i}) 
3=0 
fe-1 

= n^+i + ^i+i)~ i ( a; * i {x i+ i=! Bi } + w i {»i+i=i/ i })> 

i=0 
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making the change of variable i = k — j — 1. Dividing the two products for p and p' yields, after 
cancellation, p/p' = {xt + Vk)/{xo + Ho) = m/n, as required. □ 

Remarks. Of course by summing over paths in the equality np(u) = raj/(w) we could use the 
argument in the last proof to prove (I14p . The reversibility and the invariant measure exhibited in 
Lemma |3, 41 and (|14p will appear naturally in terms of a stationary model in Section [10. li 



4 Proof of Theorem 12.21 via the uniform renewal process 

In this section we study the asymptotic behaviour of E[Zfc + i | = n] as n — > oo. The explicit 
expression for the distribution of Z^+i given Z^ = n obtained in Lemma 13.31 turns out not to be 
very convenient to use directly. Thus we proceed somewhat indirectly and exploit a connection 
with a renewal process whose inter-arrival times are uniform on (0, 1). Here and subsequently we 
use U(0, 1) to denote the uniform distribution on (0, 1). 

Let Xi,X2>X3)--- be an i.i.d. sequence of U(0, 1) random variables. Consider the renewal 
sequence Si, i G Z + defined by Sq := and, for i > 1, Si := Ylj=i Xj- For t > define the counting 
process 

N(t) := min{i G Z+ : 5, > t} = 1 + max{i G Z + : Sj < i} , (15) 

so a.s., iV(t) > t + 1. In the language of classical renewal theory, E[iV(£)] is a renewal function (note 
that we are counting the renewal at time 0). The next result establishes the connection between 
the uniform renewal process and the simple harmonic urn. 

Lemma 4.1. For each n G N, the conditional distribution of Z^ + i on Z^ = n equals the distribution 
of N{n) — n. In particular, for n G N, E[Zfc + i | Z% = n] = E[N(n)] — n. 

The proof of Lemma 14.11 amounts to showing that P(iV(n) = n + m) = p(n, m) as given by 
Lemma 13.31 This equality is Theorem 3 in [5T], and it may be verified combinatorially using the 
interpretation of A(n,k) as the number of permutations of {1, . . . ,n} with exactly k — 1 falls, 
together with the observation that for n G N, N(n) is the position of the re th fall in the sequence 
ipl,tp2, ■ ■ ■ , where tp^ = Sk mod 1, another sequence of i.i.d. U (0, 1) random variables. Here we will 
give a neat proof of Lemma 14.11 using the coupling exhibited above in Section [3l 

Proof of Lemma 14.11 Consider a doubly-infinite sequence (£i)«ez of independent exponential 
random variables with mean 1. Taking = we can write R n ^ m (as defined in the first proof 
of Lemma [3.3p as Y17=-m^i- Define S n ^ m = Y17=-m^- ^or fixed n G N, m G Z + we consider 
normalized partial sums 




j G {1, ...,n + m}. 



Since {Sj-i- mim )™l™ are the first n + m points of a unit-rate Poisson process on E + , the vector 
(xi) X25 • • • > Xn+m) i s distributed as the vector of increasing order statistics of the n + m i.i.d. U (0, 1) 
random variables xi, ■ ■ ■ > Xn+m- In particular, 

(n+m \ /n+m \ 
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using the fact that, by (JTSJ), {N(n) > r} = {S r < n} for r G Z + and n > 0. But 

n+m m+n m / n \ 

i=l i=m+l i=l \i=—m / 



So, using the equation two lines above (fTT|) . 

P (N(n) - n > m) = P (i? n , m > 0) = P (Z fc+1 >m\Z k = n). 

Thus iV(n) — n has the same distribution as Z k+ i conditional on = n. □ 

In view of Lemma 14.11 to study E[Z/% + i | Z^ = n] we need to study E[iV(n)]. 
Lemma 4.2. As n — >■ oo, 

E[JV(n)] - ^2n + -> 0. 

Proof. This is a consequence of the renewal theorem. For a general non-arithmetic renewal process 
whose inter-arrival times have mean \x and variance a 2 , let U(t) be the expectation of the number 
of arrivals up to time t, including the initial arrival at time 0. Then 

U(t) - - -»• - + „ M , as t -> oo. (16) 

We believe this is due to Smith [38]. See e.g. Feller [TTJ §XI.3, Thm. 1], Cox [HJ §4] or Asmussen [H 
§V, Prop. 6.1]. When the inter-arrival distribution is U(0, 1), we have ?7(t) = E[iV(t)] with the 
notation of ([To]) , and in this case /U = 1/2 and a 2 = 1/12. □ 

Together with Lemma |4.H Lemma 14.21 gives the following result. 
Corollary 4.3. E[Zfc_|_i | Z k = n] — n — > | as n — > oo. □ 

To obtain the exponential error bound in (JH) above, we need to know more about the rate of 
convergence in Corollary 14.31 and hence in Lemma 14.21 The existence of a bound like for some 
oi\ < follows from known results: Stone [40] gave an exponentially small error bound in the renewal 
theorem (|16p for inter- arrival distributions with exponentially decaying tails, and an exponential 
bound also follows from the coupling proof of the renewal theorem (see e.g. Asmussen [IJ §VII, 
Thm. 2.10 and Problem 2.2]). However, in this particular case we can solve the renewal equation 
exactly and deduce the asymptotics more precisely, identifying a (sharp) value for ati in (j4]). The 
first step is the following result. 



Lemma 4.4. Let xi, X2, ■ ■ ■ be an i.i.d. sequence of U(0, 1) random variables. For t € R+, 

(t-i) k (-iy 




i\(k-i)\ ' 



i=0 



and 



E[J\T(t)] = 17(f) = £P (£><*] =^ ( * *p ■ (17) 



k=0 \i=l / i=0 
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Proof. The first formula is classical (see e.g. p. 27]); according to Feller [10, p. 285], it is due 
to Lagrange. The second formula follows from observing (with an empty sum being 0) 



u(t)=Ej2i\j2xi<t\=J2 ¥ [T / xi<t 



k=0 Ki=l ) k=0 \i=l / 

and exchanging the order in the consequent double sum (which is absolutely convergent). □ 
We next obtain a more tractable explicit formula for the expression in ()17p . Define for t > 



/(*) : = E 



w (i-t) '-»«-< 

i=0 



It is easy to verify (see also [TJ p. 148]) that / is continuous on [0, oo) and satisfies 

/(*) = e*, (0<t<l), 



/'(*) = f(t)-f(t-l), (t>l). (18) 



Lemma 4.5. For all t > 0, 

f{t) = 2t + \+ £ ±e*. (19) 

3 ' — ' 7 

7 GC: 7^0, 

7 = 1 — exp(— 7) 

TTie SHm is absolutely convergent, uniformly for t in (e, 00) /or any g > 0. 

Proof. The Laplace transform Cf{\) of / exists for Re(A) > since f(t) = 2t + 2/3 + o(l) as 
t — > 00, by (|17p and Lemma 14.21 Using the differential-delay equation (|18p we obtain 

£/(A) 



A - 1 + e- 



The principal part of Cf at is + There are simple poles at the non-zero roots of A — 1 + e _A , 
which occur in complex conjugate pairs a n ± ij3 n , where a = a.\ > Q2 > • • • and < /3i < P2 < • • • ■ 
In fact, a n = — log(27rn) + o(l) and (3 n = (2n + ^)tt + o(1) as n — > 00. For 7 = a n + ifi n the absolute 
value of the term e 7 */7 in the right-hand side of (]19p is 1/(| 7 ||1 — 7 |*), hence the sum converges 
absolutely, uniformly on any interval (e, 00), e > 0. 

To establish (|19p we will compute the Bromwich integral (inverting the Laplace transform), 
using a carefully chosen sequence of rectangular contours: 

e+iR At 



fit) = lim / ^rdA. 

R-^ooJe-iR A-l + exp(-A) 

To evaluate this limit for a particular value of t > 0, we will take e = 1/t and integrate around a 



1 



7T2. 



sequence C n of rectangular contours, with vertices at (1/t) ± (2n — ^)vrt and —2 log n ± (2n ■ 
The integrand along the vertical segment at real part — 21ogn is bounded by (1 + o(l))/n 2 and the 
integrand along the horizontal segments is bounded by e/ (2n— ^)tt because the imaginary parts of A 
and e _A have the same sign there, so |A — 1 + e _A | > Im(A). It follows that the integrals along these 
three arcs all tend to zero as n — > 00. Each pole lies inside all but finitely many of the contours C n , 
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so the principal value of the Bromwich integral is the sum of the residues of e A */ (A — 1 + exp(— A)). 
The residue at is 2t + 2/3, and the residue at 7 = a n + i/3 n is e 7 */7- Thus we obtain (fl"9l) . □ 

Proof of Theorem 12.21 The statement of the theorem follows from Lemma 14.51 since by Lemma 
IP and (dU) we have E[Z k+1 \Z k = n\= f{n) - n for n G N. □ 

Remarks. According to Feller Problem 2, p. 385], equation (|17|) "is frequently rediscovered in 
queuing theory, but it reveals little about the nature of U." We have not found the formula (|19p 
in the literature. The dominant term in f(t) — 2t — 2/3 as t — > 00 is e 7l */7i + e 7l</ 7i, he. 

2 1 a2 eait (Pi S ™(M + ai cos(Pxt)) , 
a\ + p\ 

which changes sign infinitely often. After subtracting this term, the remainder is O (e Q2 *) . The 
method that we have used for analysing the asymptotic behaviour of solutions to the renewal 
equation was proposed by A.J. Lotka and was put on a firm basis by Feller [9]; Laplace transform 
inversions of this kind were dealt with by Churchill [7]. 

5 Proof of Theorem 12.11 

The recurrence relation (I13p for r(n, m) permits a direct proof of Theorem 12. II (transience), without 
appealing to the more general Theorem 12.41 via standard martingale arguments applied to h{Z k ) 
for a judicious choice of function h. This is the subject of this section. 

Rewriting (jl3h in terms of p yields the following recurrence relation, which does not seem simple 
to prove by conditioning on a step in the urn model; for n, m G N, n + m > 3, 

n + m\ ( n \ 

p(n, m) = p(n — 1, m) + I p(n, m — 1), (20) 



m J ' \m — \ r 

where if m = 1 we interpret the right-hand side of (|20|) as just p(n — 1, 1), and where p(0, m) = 
p(n,0) = 0. Note p(l, 1) = 1/2. For ease of notation, for any function F we will write E n [i ? (Z)] 
for E[i ?1 (Zfc + i) I Zk = n], which, by the Markov property, does not depend on k. 

Lemma 5.1. Let ol\ ps —2.0888 be as in Theorem \2.!2l Then for n>2, 



E„ 
1 



Z\Z + l) 



Vn[Z]-K n _ l[ Z ] = l +o 

n n 
E n _i[l/Z]-E n [l/Z] _ 1 



n n 2 (n — 1) 



+ O (e Qin ) 



Proof. We use the recurrence relation (|20j) satisfied by the transition probabilities of Z^. First, 
multiply both sides of (|20l) by m, to get for n, m G N, m + n > 3, 



where the asymptotics refer to the limits as n — > 00. 



n 

(n + m)p(n, m) = mp(n — 1, m) + np(n, m — 1) H p(n, m — 1), 

m — 1 

where p(n, 0) = p(0, m) = 0. Summing over m G N we obtain for n > 2, 

n + E n [Z] =E n _i[Z] + n + nE n [l/Z], 
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which yields the first equation of the lemma after an application of . 

For the second equation, divide (I20p through by m to get for n, m G N, m + n > 3, 



(ra + m) 1 . n , . 

5 — p{n,m) = — p(n — 1, mH ; -p{n,m — 1) . 

m z ' m m{m — 1) 

On summing over m G N this gives, for n > 2, 

nE n [l/Z 2 ]+E n [l/Z] =E n _ 1 [l/Z]+nE n , 

which gives the second equation when we apply the asymptotic part of the first equation. □ 

Proof of Theorem 12.11 Note that h(x) = \ — ^^+1) satisfies h(n) > for all n G N while 
h{n) — > as n — > 00. By Lemma 15. II we have 

E n [h(Z)]=E n [l/Z]-E n 



ZHZ + l) 



n n z {n — 1) 



which is less than h(n) for n sufficiently large. In particular, h{Zk) is a positive supermartingale 
for Zk outside a finite set. Hence a standard result such as [U Prop. 5.4, p. 22] implies that the 
Markov chain (Zk) is transient. □ 



6 Moment and tail estimates for Zk+i — Zk 

In order to study the asymptotic behaviour of (Zfc)fcgz+; we build on the analysis of Section U] to 
obtain more information about the increments Z^+i — Zk- We write := Z^+i — Zk (k G Z + ). 
From the relation to the uniform renewal process, by Lemma 14.11 we have that 



(2n+x \ 
^2 Xi<n) , 



(21) 



where Xi>X2> • • • are i.i.d. U(0, 1) random variables, using the notation at (fT5j) . 

Lemma 16.21 below gives a tail bound for |Afc| based on (12 ip and a sharp bound for the moment 
generating function of a U(0, 1) random variable, for which we have not been able to find a reference 
and which we state first since it may be of interest in its own right. 

Lemma 6.1. For x a U(0, 1) variable with moment generating function given for A £K by 

m =E[e x *] = ^ T ±, (22) 

we have 

log<A(-A)<-^ + || (A>0); log0(A)<^ + |j (A>0). 

Proof. Consider the first of the two stated inequalities. Exponentiating and multiplying both sides 
by Ae A / 2 , this is equivalent to 

2sinh(A/2) < Aexp(A 2 /24) (23) 
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for all A > 0. The inequality f|23|) is easily verified since both sides are entire functions with non- 
negative Taylor coefficients and the right-hand series dominates the left-hand series term by term, 
because 6 n n\ < (2n + 1)! for all n G N. The second stated inequality reduces to (|23p also on 
exponentiating and multiplying through by Ae _A//2 . □ 

Now we can state our tail bound for \A k \. The bound in Lemma 16.21 is a slight improvement 
on that provided by Bernstein's inequality in this particular case; the latter yields a weaker bound 
with Ax instead of 2x in the denominator of the exponential. 



Lemma 6.2. For n G N and any integer x > we have 

P(|A fc | > x | Z k = n) < 2exp 



3x 2 



An + 2x 

Proof. From (|2ip and Markov's inequality, we obtain for x > and any A > 0, 

P(A fc > x | Z k = n) = P ^exp | - A Xi j > e " A "^ < ex P i Xn + ( 2n + x ) lo § . 

where 4> is given by (|22p . With A = 6rr/(2n + rr) the first inequality of Lemma 16. II yields 

f xX ] f 3x 2 

P(Afe > x | Zfc = n) < exp < — — > = exp 



4 J { An + 2x 

On the other hand, for x G [0,n — 1], from (|2ip and Markov's inequality once more, 

(2n-x \ / ( 2n-x ~\ \ 

^ Xi > n j = P (exp | A ^ X i| > e An j 

< exp {— An + (2n — x) log 0(A)} . 

On setting A = 6x/(2n — x), the second inequality in Lemma 16 . 1 1 yields . for any x G [0,n — 1], 

3x 2 1 f 3x 2 



P(Afc < —x | Zfc = n) < exp < — — > < exp 



An - 2x J ~ \ An + 2x J ' 

while P(Afc < — n | Z k = n) = 0. Combining the left and right tail bounds completes the proof. □ 
Next from Lemma 16.21 we obtain the following large deviation and moment bounds for A^. 

Lemma 6.3. Suppose that e > 0. Then for some C < oo and all n G N, 

P(|A fc | > n (1/2)+£ | Z k = n) < Cexp{-n e }. (24) 
/or each r G N, £/iere exists C(r) < oo suc/i i/ioi /or any n G N, 

E[|A fe | r \ Z k = n]< C(r)n r/2 . (25) 

Proof. The bound (]24[) is straightforward from Lemma 16.21 For r G N, 

/>oo 

E[| A fc | r | Z fe = n\ < / P(| Afe| > [x l / r \ \Z k = n) dx 
Jo 
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L dy < r(r/2)(2n) r / 2 r/2. 



^ C h ^\-^j dX + C l^\-—j dX > (26) 

for some C < oo, by Lemma 16.21 With the substitution y = x 1 ' r , the second integral on the last 
line of (|26p is seen to be 0(n r-1 e -n ) by asymptotics for the incomplete Gamma function. The first 
integral on the last line of (126h . with the substitution y = (2n)~ 1 x 2 ^ r , is equal to 

( 2 ^ r n/ \-yJrm- h 

Combining the last two upper bounds we verify (|25[) . □ 

The next result gives sharp asymptotics for the first two moments of = Z k+ \ — Z k . 
Lemma 6.4. Let ol\ ~ —2.0888 be as in Theorem \2.2[ Then as n — > oo, 

E[A,|Z, = n] = ^ + 0(en (27) 
E[A 2 k \Z k = n] = + ^ + 0(ne Q1 "). (28) 

Proof. The equation (|27|) is immediate from ([!]). Now we observe that J n := X 2 + Y 2 — n is a 
martingale. Indeed, for any (x,y) G I? , 

E[J n+1 - J n | (X n ,Y n ) = (x,y)} = |X| , (2j/sgn(x) + 1) + ^ (-2gsgn(y) + 1) - 1 = 0. 

\x\ + \y\ \x\ + \y\ 

Between times v k and v k+ \, the urn takes Z k + Z k+ \ steps, so Vk+i — v k = Z k + Z k+ \. Moreover, 

T "k = Z k 

71 TCf T I r 7_ 1 TI?r'72 



= Z'j- — v k . Applying the optional stopping theorem at v k and ffc+i we have that 



J Uk = Z k - v k = E[J^ fe+1 I Z k ] = E[Z k+l - v k+1 I Z k ] = E[Z fc+1 - Z k+ i \ Z k ] -v k - Z k . 

The optional stopping theorem is applicable here since a.s. J n < Cn 2 for some C < 00 and all n, 
while there is an exponential tail-bound for v k +\ — v k (see Lemma 18.11 below). Rearranging the 
equation in the last display, it follows that for n G N, 

E[Zl +l \Z k = n]=n 2 + n + K[Z k+1 \ Z k = n]. (29) 

Writing A k = Z k+ \ — Z k , we have that 

E[Al I Z k = n] = E[Z 2 k+1 I Z k = n] - 2nE[Z k+1 \Z k = n] + n 2 , 

which with ([29]) and © gives ([28]) . □ 

Remark. In view of Lemma 14.11 we could have used renewal theory (e.g., |39j ) to estimate E[A? | 
Z k = n]. However, no result we could find in the literature would yield a bound as sharp as that 
in (12511 . 

Lemma 14. II with (|27|) and (|28|) implies an ancillary result on the U(0, 1) renewal process: 

Corollary 6.5. Let N(t) be the counting function of the uniform renewal process, as defined by 
[15\) . Then with a\ « —2.0888 as in Theorem \2.2l as t — > 00, 

1 D 2 2 2 

E[N(t) 2 ] =4t 2 + —t + - + 0(te Qlt ); Var[JV(t)] = -t + - + 0(ie Qlt ). 
3 3 3 9 

These asymptotic results are both sharper than any we have seen in the literature; see e.g. [19|, I41j 
in the particular case of the uniform renewal process or [39| for the general case. We remark that 
the formula given in |41| p. 231] for E[iV(t) 2 ] contains an error (in [31] the renewal at is not 
counted, so the notation m k (-) there is equivalent to our E[(iV(-) — l) fc ]). 
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7 Asymptotic analysis of the noisy urn 



7.1 Connection to Lamperti's problem 

In this section we study the noisy urn model described in Section 12.31 To study the asymptotic 
behaviour of (Zk)k£Z + , it turns out to be more convenient to work with the process iWk)k£i + 

~ 1/2 

defined by Wk = Z k , since the latter process has asymptotically-zero drift, in a sense to be made 
precise shortly, and such processes have been well-studied in the literature. 

Let (Wk)kGZ+ be an irreducible time-homogeneous Markov chain whose state-space is an un- 
bounded countable subset of R+. Define the increment moment functions 



by the Markov property, when the corresponding moments exist the fi r (x) are genuine functions 
of x. Given a reasonable choice of scale for the process Wk, it is common that ^{x) be uniformly 
bounded away from and oo. In this case, under some mild additional regularity conditions, the 
regime where = O(l) is critical from the point of view of the recurrence classification of Wk- 

For a nearest-neighbour random walk on Z + this fact had been known for a long time (see |15j). but a 
study of this and many other aspects of the problem, in much greater generality (with absence of the 
Markovian and countable state-space assumptions) , was carried out by Lamperti [261 123 [22] using 
martingale techniques. Thus the analysis of processes with asymptotically zero drift (i.e. Hi(x) — > 0) 
is sometimes known as Lamperti's problem. 

We will next state some consequences of Lamperti's results that we will use. For convenience, 
we impose conditions that are stronger than Lamperti's. We suppose that for each r £ N, 



The recurrence and transience properties of Wk were studied by Lamperti [26 , 28j and his results 
were refined by Menshikov, Asymont and Iasnogorodskii |31j . Parts (i) and (ii) of the following 
result are consequences of Theorems 3.1 and 3.2 of [26] with Theorem 2.1 of [28], while part (iii) is 
a consequence of Theorem 3 of [31] (which is in fact a much sharper result). 

Proposition 7.1. [2h\ \28 { [31 ^ Let (Wk) be an irreducible Markov chain on a countable unbounded 
subset o/R+. Suppose that (31\) holds, and that there exists v > such that ^{x) > v for all x 
sufficiently large. Then the following recurrence criteria are valid. 

(i) Wk is transient if there exist S,xq 6 (0,oo) such that for all x > xq, 



fir(x) := E [(W fc+ i - Wk) r \W k = x}; 



(30) 



sup |/u r (x)| < oo. 



(31) 



.r 



2x/j,i(x) — ^2{x) > 5. 



(ii) Wk is positive-recurrent if there exist 5,xq € (0, oo) such that for all x > xq 



2x/j,i(x) + H2{x) < — & 



(iii) Wk is null-recurrent if there exists xq G (0, oo) such that for all x > xq 
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In |27j . Lamperti proved the existence of weak-sense limiting diffusions for certain processes 
satisfying parts (i) or (iii) of Proposition 17.11 To state Lamperti's result we need some more 
notation. To describe the time-homogeneous diffusions on R + that arise here, it will suffice to 
describe the infinitesimal mean n(x) and infinitesimal variance o~ 2 (x); see e.g. |214 Ch 15]. The 
transition functions p of our diffusions will then satisfy the Kolmogorov backwards equation 

dp f N dp i 1 2 N d 2 p 



Let (Hf )te[o,i] denote a diffusion process on R + with infinitesimal mean and variance 



a 



H{x) = -, a 2 (x) = (3. (32) 



x 



The particular case of a diffusion satisfying (|32p with (3 = 1 and a = (7 — l)/2 for some 7 E R 
is a Bessel process with parameter 7; in this case we use the notation V? = 1)72,1 . Recall 
that for 7 G N, the law of (V^ y )te[o,il is the same as that of ||-£?t llte[o,i] where (-Bt)tg[o,i] is standard 
7-dimensional Brownian motion. In fact, any H^'^ is related to a Bessel process via simple scaling, 
as the next result shows. 



Lemma 7.2. Let a & R and (3 > 0. TTie diffusion process H^^ is a scaled Bessel process: 
{Ht'^)t<a\Q,i\ has the same law as {0 l ^Vt)te[Q,i\i with 7 = 1 + — . 

Proof. By the Ito transformation formula (cf p. 173 of [21]). for any (3 > the process (/3 1//2 V^ 7 ) tg [ ji 
is a diffusion process on [0, 1] with infinitesimal mean fi(x) = (3( r y—l)/(2x) and infinitesimal variance 
o~(x) = {3, from which we obtain the result. □ 

We need the following form of Lamperti's invariance principle |27[ Thms. 2.1, 5.1, and A. 2]. 



Proposition 7.3. f27^ Let (Wk) be an irreducible Markov chain on a countable unbounded subset 
o/R + . Suppose that j[3~T\) holds, and that 



lim fi2(x) = (3 > 0, lim xfj,i(x) = a > —{(3/2). 

x— >oo x— >oc 

Let {Ht^)t>Q be a diffusion process as defined at II Sty). Then as k — > 00, 

in the sense of convergence of finite- dimensional distributions. Marginally, 

lim Flk^Wk <y) = , t^tttt^ 2 , ; ; t^tt T r 2 ^ exp(-r 2 /(2^))dr. 

fe^oo v fe - y; (2/3)W/ 3 )+( 1 /2)r(( a //3) + (1/2)) y PV /v 



7.2 Increment moment estimates for Wf. 

~ 1/2 

Now consider the process (Wk)keZ + where Wk = Z k ; this is a Markov chain with a countable 
state space (since Z k is), so fits into the framework described in Section [7TT1 above. Lemma [7771 
below shows that indeed Wk is an instance of Lamperti's problem in the critical regime. First we 
need some simple properties of the random variable n. 
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Lemma 7.4. If k satisfies for A > 0, then 

P(\k\ >x)< exp{-Ax}, (x > 0), and (33) 
E[\k\ t ] < oo, (r > 0). (34) 

Proof. (|33p is immediate from Markov's inequality and ([5]), and (|34p is also straightforward. □ 

Now we can start our analysis of the noisy urn and the associated process Z k . Recall that Z k is 
defined as max{|Xj> fc+ i|, | Yp fc _|_i|}. By definition of the noisy urn process, if we start at unit distance 
away from an axis (in the anticlockwise sense) , the path of the noisy urn until it hits the next axis 
has the same distribution as the corresponding path in the original simple harmonic urn. Since we 
refer to this fact often, we state it as a lemma. 

Lemma 7.5. Given Z k = z, the path (X n ,Y n ) for n = v k + 1, . . . , v k +\ has the same distribution as 
the path (X n , Y n ) for n = v k + 1, . . . , Vk+i given Z k = z. In particular, Z k+ i conditioned on Z k = z 
has the same distribution as Z k+ \ — min{K, Z k+ \ — 1} = Z k+ \ — k + (k + 1 — Z k+ i)l{n > Z k+ {\ 
conditioned on Z k = z. 

Recall that A k = Z k+ ± — Z k , and write A k = Z k+ \ — Z k . The next result is an analogue of 
Lemmas 16.31 and 16.41 for A^.. 

Lemma 7.6. Suppose that |5|) holds. Let e > 0. Then for some C < oo and all n € N, 

PflAjfel > n^ +£ \Z k = n)< Cexp{-n e/3 }. (35) 
Also, for any r € N, there exists C < oo such that for any n £ N, 

E[|A fc | r \Z k = n]< Cn r ' 2 . (36) 
Moreover, there exists 7 > for which, as n — > 00, 

E[A k I Z k = n] = I - E[k] + 0(e~^), (37) 
E[A 2 k I Z k = n] = l n + 0(1). (38) 

Proof. By the final statement in Lemma 17.51 for any r > 0, 

P(|A fc | >r\Z k = n)< P(\A k -n\>r\Z k = n). 

We have for any e > 0, 

P(|A fe -k\> n^ 2)+e \Z k = n)< P(|A fc | > n {1+£ ^ 2 \ Z k = n) + P(|k| > n {1+£ ^ 2 ), 

for all n large enough. Using the bounds in (j24"l) and ([33]) we obtain (j35j) . For r e N, 

E[|A fc | r I Z k =n]< E[(\A k \ + \ K \) r | Z k = n]. 

Then with Minkowski's inequality, (|25p and (|34p we obtain (|36|) . 
Next we have from Lemma 17.51 and (I27p that 

E[A k \Z k = n]= E[A k _« + (« + !- Z k+1 )1{ K > Z k+1 }} \ Z k = n] 



2G 



= - + 0(e Qin ) - E[«] + E[(k + 1 - Z k+1 )1{K > Z k+1 } \Z k = n}. 

By the Cauchy-Schwarz inequality, (|34p and the bound < k + 1 — ^fc+i < k, the last term here 
is bounded by a constant times the square-root of 

F(k > ^fc+i I ^fc = n) < P(|A fe | > n/2 \Z k = n) + P(|/c| > n/2) = 0(exp{-An/2}), 

using the bounds (j24"l) and (i33l) . Hence we obtain (|37|) . Similarly, from (j28l) . we obtain ([38]) . □ 

Now we can give the main result of this section on the increments of the process (W k ) k £z + . 
Lemma 7.7. Suppose that k satisfies |5|). With // r (x) as defined by [W\). we have 

sup |/U r (x)| < oo, (39) 



for each r G N. Moreover as x — > oo, 

1 - 2E[/e] 



4x 



+ 0(x~ 2 ); /i 2 (x) = - + 0(x- 1 ). 

o 



(40) 



Proof. For the duration of this proof we write E x 2 [ ■ ] for E[ • | Z k = x 2 ] = E[ • | W k = x]. For r G N 
and x > 0, from (l30l). 



Mr (x)| < E x2 [\zl^ - Zl /2 \ r ] = x r E x2 |(1 + x~ 2 A fc ) 1/2 - 1 



(41) 



Fix e > and write A(n) := {\A k \ > n^/ 2 ^ +e } and J 4 c (n) for the complementary event. Now for 
some C < oo and all x > 1, by Taylor's theorem, 



(1 + x~ 2 A fe ) 1/2 - l\ r U^) < Cx- 2r \A k \ r l Ac{x2) . 



Hence 



E, 



|(l + ar 2 A fc ) 1/z -iri 



A c (x 2 ) 



< Cx- 2r E^\\A k 



0(x~ r ), 



(42) 



by ()36p . On the other hand, using the fact that for y > —1, < (1 + y) 1 ^ 2 < 1 + (y/2), we have 



E x 2 



i + x^Afc) 1 / 2 - in 



A(x 2 ) 



< E.,2 



l + (l/2)x- 2 |A A 



l A{x 2 ) 



< (E^l + |A fc |) 2 nV /2 fp(,4(x 2 ) I Z k = x 2 )' " 



for x > 1, by Cauchy-Schwarz. Using (|36j) to bound the expectation here and (I35j) to bound the 
probability, we obtain, for any r G N, 



E„2 



|(1 + x" 2 A,) 1/2 - l| r W)l = 0(exp{-x £ / 2 }). 



A{x 2 ) 

Combining (g2|) and (@3]) with (JH]) we obtain ([39]) . 
Now we prove (|40p . We have that for x > 0, 

M(x) = E x2 [W k+1 - W k ] = xE x2 [(l + x" 2 A fc ) 1 /2 _ i] 

= xE x2 [((1 + x~ 2 A,.) 1 / 2 - l)l A c (x2) ] + 0(exp{-x £ / 3 }), 



(43) 



(44) 
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using (|43p . By Taylor's theorem with Lagrange form for the remainder we have 
xE x2 [({l + x^Ak) 1 / 2 - l)l Ac(x2) ] 

= ^-E a , 2 [A,l^)] - ^E x 2[A 2 k l AHx2) ] + 0(x- 5 E, 2 [|A fe | 3 ]). (45) 

Here we have that x K x 2 [| A k \ 3 ] = 0(x~ 2 ), by ()36j) . while for r E N, we obtain 

E^l^] =E x2 [A r k ] +0((E a;2 [|A fc | 2 nP(^(x 2 ) | Z k = x 2 )) 1 ' 2 ), 
by Cauchy-Schwarz. Using (|35p again and combining (|44p with (|45[) we obtain 

= ^E, 2 [A fc ] - ^3E, 2 [A 2 fc ] + 0(x~ 2 ). 

Thus from f)3T[) and (|38p we obtain the expression for /ii in (|40p . Now we use the fact that 
(W k+l - W k ) 2 = W 2 +1 - W 2 - 2W k (W k+1 - W k ) = Z k+l -Z k - 2W k (W k+1 - W k ) 
to obtain ^(x) = E x2 [Aj.] — 2x/ii(x), which with (|37p yields the expression for fi 2 in (|40p . □ 

8 Proofs of theorems 

8.1 Proofs of Theorems E31 EH, EHi and 

First we work with the noisy urn model of Section \2. 31 Given the moment estimates of Lemma [7. 71 
we can now apply the general results described in Section [7.11 and [3]. 

Proof of Theorem 12.41 First observe that (Z k ) kG x + is transient, null-, or positive-recurrent 
exactly when (W k ) k ^z + is- From Lemma l7,7( we have that 

1 2 

2x^(x) - fi 2 (x) = -- E[k] + 0(x _1 ); 2xfi 1 (x) + /j, 2 (x) = -- E[k] + 0{x~ l ). 
o o 

Now apply Proposition 17.11 □ 

Proof of Theorem 12.51 By the definition of r q at ([6]), T q is also the first hitting time of 1 
by (Wfc)fcgN- Then with Lemma 17.71 we can apply results of Aspandiiarov, Iasnogorodski and 
Menshikov [31 Props. 1 and 2], which generalize those of Lamperti [28] and give conditions on /xi 
and [i 2 for existence and non-existence of passage-time moments, to obtain the stated result. □ 

Proof of Theorem 12.81 First Proposition 17.31 and Lemma 17.71 imply that, as n — > oo, 

(n- l / 2 W nt ) tem -)■ (#r%o,i], 

in the sense of finite-dimensional distributions, where a = (1 — 2E[k])/4 and f3 = 1/6, provided 
E[k] < 2/3. By the ltd transformation formula (cf p. 173 of [21]), with as defined at ()32|) . 

(H^^) 2 is a diffusion process with infinitesimal mean fi(x) = (5 + 2a and infinitesimal variance 
cr 2 (x) = Af3x. In particular (H^ 13 ) 2 has the same law as the process denoted Dt in the statement of 
Theorem 12.81 Convergence of finite-dimensional distributions for (n~ 1 W 2 t ) t& [Q^] = {n~ l Z nt )t^[o,i] 
follows. The final statement in the theorem follows from Lemma 17.21 □ 
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Next consider the leaky urn model of Section 12.21 

Proof of Theorem 12. 3L This is an immediate consequence of the P(k = 1) = 1 cases of Theorems 
iLlandELl □ 



Remark. There is a short proof of the first part of Theorem 12.31 due to the existence of a particular 
martingale. Consider the process Q' n defined by Q' n = Q(X' n ,Y^) where 

Q(x, y) ■= {^ + \ s S n (y) - ^{^o} sgn(x)^j + {y~\ s ^(x) - ^l{x=o} sgn(y)^ . 

It turns out that Q' n is a (non-negative) martingale. Thus it converges a.s. as n — > oo. But since 
Q(x,y) — > oo as ||(x,y)|| — > oo, we must have that eventually (X' n ,Y^) gets trapped in the closed 
class C. So P(t < oo) = 1. 



8.2 Proofs of Theorems EH and EH 



The proofs of Theorems 12.61 and 12.91 that we give in this section both rely on the good estimates 
we have for the embedded process Z k to analyse the noisy urn (X n ,Y n ). The main additional 
ingredient is to relate the two different time-scales. The first result concerns the time to traverse a 
quadrant. 

Lemma 8.1. Let k € Z+. The distribution of v k +\ — & k given Z k = n coincides with that of 
Z k+ i + Z k given Z k = n. In addition 

h+i ~ Vk = \Xy k+1 1 + \% k+1 \+Z k . (46) 

Moreover 

P(i> fc +i - v k >3n\Z k = n) = Otexpj-n 1 / 2 }). (47) 



Proof. Without loss of generality, suppose that we are traversing the first quadrant. Starting at 
time V}. + 1, Lemma [7. 51 implies that the time until hitting the next axis, v k +\ ~ v k — 1, has the same 
distribution as the time taken for the original simple harmonic urn to hit the next axis, starting 
from {Z k , 1). In this time, the simple harmonic urn must make Z k horizontal jumps and Z k +i — 1 
vertical jumps. Thus v k +i ~ & k — 1 has the same distribution as Z k+ \ + Z k — 1, conditional on 
Z k = Z k . Thus we obtain the first statement in the lemma. For equation (]46p . note that between 
times v k + 1 and v k +i the noisy urn must make Z k horizontal steps and (in this case) |^i> fe+1 | — 1 
vertical steps. Finally we have from the first statement of the lemma that 

P(P fc+ i - v k > 3n | Z k = n) = ^{Z k+ i > 2n | Z k = n), 

and then (@7D follows from (JMJ) ■ □ 

Very roughly speaking, the key to our Theorems 12.61 and 12.91 is the fact that r w SfeLo ^1 
and A « Ylk=o ^fc ■ Thus to study r and A we need to look at sums of powers of W k over a 
single excursion. First we will give results for S a := ^fcLo^T' Q — 0- Then we quantify the 
approximations '~' for r and A by a series of bounds. 

Let M := maxo<Kr, W k denote the maximum of the first excursion of W k . For ease of notation, 
for the rest of this section we set r := 6E[k] — 3. 
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Lemma 8.2. Suppose that r > — 1. Then for any e > 0, for all x sufficiently large 
In particular, for any e > 0, E[M 1+r+e ] = oo 6u£ E[M 1+r ~ e ] < oo. 

Proof. It follows from Lemma 17.71 and some routine Taylor's theorem computations that for any 
e > there exists wo G [1, oo) such that for any x > wq, 

nw l k tl +e - Wl +r+£ \W k = x}>0, 

nwixr e - wi +r ~ e i w k = x ] < o. 

Let rj := min{A; G Z + : W k < wq} and a x := min{A; G Z + : > x}. Recall that (W / fc)fcez + is 
an irreducible time-homogeneous Markov chain on a countable subset of [l,oo). It follows that to 
prove the lemma it suffices to show that, for some w > 2wq, for any e > 0, 

a.-1-r-e < < v | ^ = u; ) < x -l-r+e j ( 48 ) 

for all x large enough. 

We first prove the lower bound in (jig]) . Fix x > u?. We have that M^.^T^ is a submartingale, 
and, since is an irreducible Markov chain, rj < oo and <7 X < oo a.s.. Hence 

F(a x < 7])K[W^+ r+£ ] + (1 - F(a x < 7]))E[W^ +r+£ ] > w 1+r+£ . 

Here W n < t^o a.s., and for some C G (0,oo) and all x > w, 

nw£ r+£ ] < E [(x + - W-^-O) 1 ^] < Cx 1+r+£ , 

since E[(W CT:I . — W / a- a; -i) 1+r+£ ] is uniformly bounded in x, by equation (|39p . It follows that 

F{a x < n) [Cx 1+r+£ - wl +r+£ ] > w 1+r+£ - wl +r+£ > 0, 



which yields the lower bound in (|48p . The upper bound follows by a similar argument based on 
the super martingale property of W k jZ^ £ x . □ 

The next result gives the desired moment bounds for S a . 
Lemma 8.3. Let a > and r > -1. Then E[Sg] < oo i/p < g§ and E[Sg] = oo i/p > g§. 

Proof. First we prove the upper bound. Clearly S a < (1 + r q )M a . Then, by Holder's inequality, 

E[S£] < (E[(l + T g ) (2+Q)p/2 ])^(E[M( 2+Q ) p ])^. 

For p < ±±§ we have (2 + a)p/2 < (1 + r)/2 = 3E[«] - 1 and (2 + a)p < 1 + r so that Lemma ED 
and Theorem 12.51 give the upper bound. 

For the lower bound we claim that there exists C G (0, oo) such that 

F(S a > x) > -P(M > Cz=T*), (49) 
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for all x large enough. Given the claim (|49p . we have, for any e > 



E[Sg] > 



P 



P 



, l + r 

x p x <*+ 2 dx, 



by Lemma 18.21 Thus E[5S] = oo for p > It remains to verify (|49p . Fix y > 2. Let 

J 7 ^ = cr(Wi, . . . , Wfc), and define stopping times 

(Ti = min{fc E N : Wfc > y}; a 2 = min{/c > cti : W k < y/2}. 

Then {o~i < r q }, i.e., the event that W k reaches y before 1, is J r ai -measurable. Now 

P ({a x < r q } n {(j 2 > ai + 5y 2 }) = E [l{a x < r ? }P(<7 2 >a x + 5y 2 \ F ai )] . 

We claim that there exists 5 > so that 



1 



(50) 
(51) 



P(<72>ai + V I > -, a.s. 
Let D k = (y- W k ) 2 l{W k < y}. Then, with A k = W k+1 - W k , 

E[D k+1 - D k | JF k ] < 2(W k - y)E[A k \ F k ] + E[A|| F t \. 

Lemma E7] implies that on {W k > y/2} this last display is bounded above by some C < oo not 
depending on y. Hence an appropriate maximal inequality [32^ Lemma 3.1] implies (since D ai = 0) 
that P(maxo< s <fc -D( - 1+s )Ao- 2 > w ) < Ck/w. Then, since D a2 > y 2 /4, we have 



\<J2 <<y 1 + 5y 2 \F ai )<^( max D ((T1+s)A(T2 > (y 2 /4) | jO < 

yl<s<dj/ 2 J 



C5y 2 1 

< -, a.s., 



(2/2/4) " 2 

for 5 > small enough. Hence (|5ip follows. Combining (|50p and (|5ip we get 

P ({(Ti < rj n {a 2 > (7! + 5y 2 }) > ip(<7i < T q ) = ip(M > y). 
Moreover, on {a\ < T q } n {c 2 > o"i + 5y 2 } we have that W s > y/2 for all o"i < s < a 2 , of which 



□ 



there are at least by values; hence S a > 5y x (y/2) a . Now taking x = 2 a 5y +a we obtain (j39 
and so complete the proof. 

Next we need a technical lemma. 

Lemma 8.4. Let p > 0. Then for any e > there exists C < oo suc/i i/iai 



E 



E 



< CE[rP +e ]. 



(52) 



Proof. For any s G (0, 1) 



X>*J >x <P(r g >x s ) + P ]T| 



\Jfe=l 



a-=i 



For any random variable X, E[X P ] = p J °° x p-1 P(X > x)dx <l+pf™ a^PpT > x)dx; so 



E 



E 



/oo poo / X s \ 

x^P^g > x s )dx +p y x p-1 P \ J2 I^J > X J dx - ( 53 ) 
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Here we have that 

(X s \ / X s \ X s 

fc=l / \fc=l / k=l 

by Boole's inequality. Then Markov's inequality and the moment bound ([5]) yield 



J2\K0 k \ >x) <x s P(e A|K| >e xl ') < x s E[e x ^]e~ x1 ' 8 . (54) 



It follows that, since s < 1, the final integral in (|53p is finite for any p. Also, from Markov's 
inequality, for any e > 0, 

/oo roo 
xP-^Tg > x s )dx < E[tP +£ ] J xP-^P+^dx; 

taking s close to 1 this last integral is finite, and f|52f) follows (noting r q > 1 by definition). □ 
Proof of Theorem 12. 6L By the definitions of r and r q we have that r = i> Tq = — 1 + 5ZfcLi(^fc ~~ 



i), recalling vq = — 1. Hence by Lemma 18.14 

r = -l + £(Wg_i + W2) + i2, 

fe=i 

for i? a random variable such that \R\ < Ylk=l \ K v k \- It follows that 

Tq T q 

-\ + Y J Wl-\R\<r<2Y J Wl + \R\. (55) 

fc=0 k=0 

Lemma 18.41 implies that for any e > there exists C < oo such that E[|i?| p ] < CK[r q +£ ]. The 
E[«] > 1/3 case of the theorem now follows from (|55p with Theorem l2.5t Lemma[8]3]and Minkowski's 
inequality. In the E[k] = 1/3 case, it is required to prove that E[r p ] = oo for any p > 0; this follows 
from the E[/c] = 1/3 case of Theorem 12.51 and the fact that r > r q a.s.. □ 

Proof of Theorem 12.91 First note that we can write 



n=l n=l k=l 



where Ay = Yln=i ano - -^fc = Sn=5 i+i — ^) * s * ne area swe Pt out in traversing a quadrant 
for the kth time. Since > 1/2, part (i) of the theorem is immediate from part (i) of Theorem 
12.41 For part (ii), we have that 

A k < (z k + |«i>j)(ir fc _i + l/cp^J) 

Thus 

fc=0 
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where R x = E?=i W?-il^J> R 2 = El ? =i WfcK-i I and R 3 = E^Li l^-ill^J- Here E k =o W k 
has finite pth moment for p < 3E M -1 ; by Lemma 18.31 Next we deaf with the terms R\ , R2 and R3 . 
Consider R\. We have that, by Holder's inequality, E[|i?i| p ] is at most 



E 



3p/2 



\k=l 



2/3 



E 



\k=0 



3p 



1/3 



< C'E 



r q \ 3p/2' 

r2 



5>l 



\k=0 



2/3 



E[r„ 3p+e ] 1/3 , 



for any e > 0, by (|52|) . Lemma 18.31 and Theorem 12.51 show that this is finite provided p < 3 -^^ — - 
(taking e small enough). A similar argument holds for R2. Finally, 



E[|R 3 | P ] < E 



El 



2p 



< C"E[t 2p+£ ], 



for any e > 0, by (|52p . For e small enough, this is also finite when p < — 1 by Theorem 12.51 



These estimates and Minkowski's inequality then complete the proof. 



□ 



8.3 Proof of Theorem I27TT1 

We now turn to the percolation model described in Section 12.41 

Lemma 8.5. Let v and v' be any two vertices of G. Then with probability 1 there exists a vertex 
w G G such that the unique semi-infinite oriented paths in H from v and v' both pass through w. 

Proof. Without loss of generality, suppose v, v' are distinct vertices in G on the positive x-axis 
on the same sheet of 7Z. Let Zq = \v\ < Z' = \v'\. The two paths in H started at v and v', call 
them P and P' respectively, lead to instances of processes Z k and Z(, each a copy of the simple 
harmonic urn embedded process Z k . Until P and P' meet, the urn processes they instantiate are 
independent. Thus it suffices to take Z k , Zt to be independent and show that they eventually cross 
with probability 1, so that the underlying paths must meet. To do this, we consider the process 
(H k ) k£ % + defined by H k := \J~Z k ~ — \fZ~k and show that it is eventually less than or equal to 0. 

For convenience we use the notation Wk = (Zk) 1 / 2 and W' k = (Z' k ) 1 / 2 . Since H k+ i — Hk = 
{W' k+1 - W' k ) - (W k+1 - W k ), we have that for x < y, 

E[H k+1 -H h \W k = x,Wt = v] = ± j -± + 0{x~ 2 ) = -fc^ + 0(x-% 

by the E[k] = case of (|40p . Similarly, 

E[(H k+1 - H k ) 2 \ W k = x,W' k = y] = ^ + 0{x~ l ), 

from (|4U|) again. Combining these we see that 

2(y - x)E[H k+1 -H k \W k = x 1 W' k = y\- E[(H k+1 - H k ) 2 \W k = x,Wj e = y] 

<-^ + 0(x~ 1 ) <0, 
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for x > C, say. However, we know from Theorem 1 2 . 1 1 that Wk is transient, so in particular > C 
for all k > T for some finite T. Let r = mm{k G Z + : < 0}. Then we have that Hkl{k < r}, 
k > T, is a process on M+ satisfying Lamperti's recurrence criterion (cf Proposition 17. ip . Here 
-fffcAr is not a Markov process but the general form of Proposition 17.11 applies (see (26J Thm. 3.2]) 
so we can conclude that P(r < oo) = 1. □ 

Lemma 8.6. The in-graph of any individual vertex in H is almost surely finite. 

Proof. We work in the dual percolation model H' . As we have seen, the oriented paths through 
H' simulate the leaky simple harmonic urn via the mapping The path in H 1 that starts from 
a vertex over (n + 1/2, 1/2) explores the outer boundary of the in-graph in H of a lift of the set 
{(«, 0) : 1 < i < n}. The leaky urn a.s. reaches the steady state with one ball, so every oriented 
path in H' a.s. eventually joins the infinite path cycling immediately around the origin. It follows 
that the in-graph of any vertex over a co-ordinate axis is a.s. finite. For any vertex v of H, the 
oriented path from v a.s. contains a vertex w over an axis, and the in-graph of v is contained in 
the in-graph of w, so it too is a.s. finite. □ 

All that remains to complete the proof of Theorem l2.11l is to establish the two statements about 
the moments of I(v). For p < 2/3, E[/(v) p ] is bounded above by E[^4 P ], where A is the area swept 
out by a path of the leaky simple harmonic urn, or equivalently by a path of the noisy simple 
harmonic urn with P(k = 1) = 1 up to the hitting time r. E[^4 P ] is finite, by Theorem 12. 9f ii). The 
final claim E[/(u)] = oo will be proven in the next section as equation (|60p . using a connection with 
expected exit times from quadrants. 



9 Continuous- time models 

9.1 Expected traversal time: proof of Theorem 12.141 

Proof of Lemma 12.121 A consequence of Dynkin's formula for a continuous-time Markov chain 
X(t) on a countable state-space S with infinitesimal (generator) matrix Q = (qij) is that for a 
function g : M + x S — > R with continuous time-derivative to be such that g(t,X(t)) is a local 
martingale, it suffices that 

^^1 + Q( g (t r ))(t,x)=0, (56) 

for all x e S and t G M+: see e.g. [371 p. 364]. In our case S = C\ {0}, X(t) = A(t) + iB(t), and 
for z = x + iy G C, 

QU)[z) = E W/H " /(*)] = \x\\f(z + sgn(x)i) - /(*)] + \y\[f(z - sgn(y)) - /(*)]. 

Taking f(x + iy) = g(t,x + iy) to be first xcosi + ys'mt and second ycost — xsini we verify the 
identity (|56[) in each case. Thus the real and imaginary parts of Mt are local martingales, and 
hence martingales since it is not hard to see that E|A(i) + B(t)\ < oo. □ 

To prove Theorem 12.141 we need the following bound on the deviations of Tf from tt/2. 

Lemma 9.1. Suppose e n > and e n — > as n — >■ oo. Let (fr n G [0, vr/2]. Then as n —> oo, 

P ( T/ ~ \ + < ^ n ~ &n ' A ^ = ncos ^' jB (°) = ™sin^ n ) = Oin^e' 2 ), 
uniformly in (4> n )- 
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Proof. First note that Mq = ne*^™ and, by the martingale property, 

E[\M t - M \ 2 } = E[\M t \ 2 ] - |M | 2 = E[A(t) 2 + B(t) 2 } - n 2 . 
We claim that for all n £ N and t e R + , 

E[A(t) 2 + B{t) 2 ] - n 2 < j + 2 l / 2 nt. (57) 

Since Mt — Mg is a (complex) martingale, \Mt — Mq\ 2 is a submartingale. Doob's maximal inequality 
therefore implies that, for any r > 0, 

P ( sup \M S - M | > r ) < r~ 2 E[|M t - M | 2 ] < 2i(i + n)r" 2 , 

\0<s<t ) 

by (153). Set t = (vr/2) - n + for 6» G (0, vr/2). Then on {t < T/}, A(t ) + *-B(*o) nas argument 
in [^ n ,7r/2], so that Mt has argument in [2(p n — (vr/2) — 6,(j> n — 6]. All points with argument in 
the latter interval are at distance at least nsin# from Mq. Hence on {i < tj}, 

sup \M S - M 1 > |M to - M 1 > nsin6». 

0<s<t 

It follows that for e n > with e n — > 0, 

P(r/ > (vr/2) -<£„ + £„) < P I sup \M S - M | > n sme n I 

\0<s<(i-/2)-<}!>„+£n / 

= ©(n-^sin^)" 2 ) = O^"^^ 2 ). 

A similar argument yields the same bound for P n (7/ < (vr/2) — 4> n — e n ). It remains to prove the 
claim (|57p . First note that 

E[A(t + At) 2 + 5(t + At) 2 - (A(t) 2 + S(t) 2 ) | A(t) = x, B(t) = y] = (\x\ + \y\)At + 0((At) 2 ), 

and + \y\) 2 < 2(x 2 + y 2 ). Writing g{t) = E[A(t) 2 + B(t) 2 ], it follows that 

±g(t) < V^git) 1 / 2 , 

with g(0) = n 2 . Hence git) 1 / 2 <n + 2~ 1 / 2 t. Squaring both sides yields (|57]) . □ 

A consequence of Lemma [9. II is that tj has finite moments of all orders, uniformly in the initial 
point: 

Lemma 9.2. For any r > there exists C < oo such that max ne pj E n [rj] < C. 

Proof. By Lemma l9.1( we have that there exists no < oo for which 

sup P(t/ - t > 2n | A(t) +iB(t) = x + iy) < 1/2. (58) 

x>0,j/>0:|3;+ij/|>no 

On the other hand, if \A(t) + iB(t)\ < hq, we have that rj — t is stochastically dominated by a 
sum of no exponential random variables with mean 1. Thus by Markov's inequality, the bound (|58p 
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holds for all x > 0,y > 0. Then, for t > 1, by conditioning on the path of the process at times 
2no, 4no, ■ ■ ■ , 2no(t — 1) and using the strong Markov property we have 

t-l 

P n (r/ > 2n t) < ] sup P(r f - 2n j > 2n | A(2n j) + iB(2n j) = Xj + iyj) < 2 l ~\ 

j=1 x 3 >0, yj >0 

by (|58j) . Hence P^?/ > decays faster than any power of t, uniformly in n. □ 

Proof of Theorem I2TT41 For now fix n £ N. Suppose A(0) = Z = n, B(0) = 0. Note 
that A(tj) = 0, B(rf) = Z\. The stopping time tj has all moments, by Lemma 19.21 while 
E n [|M t | 2 ] = 0(t 2 ) by (J57]), and E n [|M T/ | 2 ] = E„[Z 2 ] < oo. It follows that the real and imaginary 
parts of the martingale Mf AT/ are uniformly integrable. Hence we can apply the optional stopping 
theorem to any linear combination of the real and imaginary parts of Mt AT/ to obtain 

E n [Zi (a sin 77 + /3cosr/)] = an 

for any a, (5 £l. Taking a = cos 9, (3 = sm.9 this says 

ncos9 = E n [Zi sin(0 + r/)] = E n [(Zi - E n Z0 sin(fl + r/)] + E n [Zi]E n [sin(0 + r/)], 

for any 9. By Cauchy-Schwarz, the first term on the right-hand side here is bounded in absolute 



value by y / Var n (Zi), so on re-arranging we have 



„ r ,„ m ncos9 



(Var^)) 1 / 2 (E n [A 2 ])V2 



E n [Zi] " E»[Zi] ' 

and then using ([2"7|) and (|28p we obtain, as n — )• oo, 

|E n [sin(# + T/)] -cos0| =0(n" 1/2 ), (59) 

uniformly in 9. This strongly suggests that tj is concentrated around 7r/2, 5tt/2, To rule out 

the larger values, we need to use Lemma [97TI We proceed as follows. 

Define the event E n := {\tj — (tt/2)| < £„} where e n — > 0. From the 9 = —ir/2 case of (|59p we 
have that E n [sin(rj — (-7r/2))] = (^(n" 1 / 2 ). Since sin 2; = x + 0(x 3 ) as x — )• we have 

E n [l En sm(r f - (tt/2))] = E^l^J - | + O(^) + 0(P n (^)) 
= E n [ T/ ] - I + 0(4) + 0((E n [T / 1) 1 /-(P n (^))i-(Vr) )5 

for any r > 1, by Holder's inequality. Here E n [rJ] = O(l), by Lemma 19.21 so that for any (5 > 0, 
choosing r large enough we see that the final term in the last display is 0(n 6 ~ 1 e~ 2 ) by Lemma 19. 11 
Hence for any 5 > 0, 

0{n^ 2 ) = E n [r f ] - I + Oin 5 -^- 2 ) + 0(e 3 n ) + E n [l E c sin(r / - (tt/2))], 

and this last expectation is 0(n~ 1 e~ 2 ) by Lemma 19.11 once more. Taking e n = n" 1 / 4 yields ([7]). 
Next, from the 9 = case of ([55])) we have that E n |1 - cos(t/ - (vr/2))| = C^n -1 / 2 ). This time 



E n [|1 - co S (r/ - (n/2))\ 1 E J = E n [|r/ - (7r/2)|^J + 0(e 4 ). 
Following a similar argument to that for ([7]), we obtain ([H]). □ 
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9.2 Traversal time and area enclosed: proofs of Theorems 12.101 and 12.151 



Our proofs of Theorems 12.101 and 12.151 both use the percolation model of Section [27 

Proof of Theorem 12.151 The asymptotic statement in the theorem is a consequence of Theo- 
rem 12.141 Thus it remains to prove the exact formula. For x > 0, and y > 0, let T(x, y) denote 
E[r/ | A(0) = x,B(0) = y]. Also, set T(0,y) = for y > 0. Note that T(n,0) = E n [r/]. Condition- 
ing on the first step shows that for x > and y > 0, 

T(x, y) = + T(x, y + l) + -^—T(x -l,y). 

x + y x + y x + y 

For fixed x, T(x,y) — > as y — > oo. Indeed, for y > 1 the time to make x horizontal jumps is 
stochastically dominated by the sum of x exponential random variables with mean 1/y. 

We now consider the percolation model restricted to the first quadrant. More precisely, we 
consider the induced graph on the set of sites (x, y) with x > and y > 0, on a single sheet of 1Z. 
Let I(x, y) denote the expected number of sites in the in-graph of (x, y) in this restricted model. 
This count includes the site (x, y) itself. For x > we also set I(x, 0) = 0. Considering the two 
possible directed edges into the site (x,y), we obtain 

I(x, y) = l + —^—I(x + l,y)+ * - J(a, y-l). 
x + y + 1 x+y - 1 

Dividing through by (x + y) we have 

I(x,y) _ 1 y f I{x + 1, y) \ | x fl{x,y-l) 



x + y x + y x + y\(x + l) + yj x + y \x + (y — 1 

We now claim that for each fixed y, I(x,y) is bounded as x — > oo. Indeed, the number of sites in 
the in-graph of (x, y) is at most y plus y times the number of horizontal edges in this in-graph. 
The number of horizontal edges may be stochastically bounded above by the sum of y geometric 
random variables with mean 1/x, so its mean tends to as x — > oo. 

We see that I(y,x)/(x + y) and T(x,y) satisfy the same recurrence relation with the same 
boundary conditions; their difference satisfies a homogeneous recurrence relation with boundary 
condition at x = and limit as y — > oo for each fixed x. An induction with respect to x shows 
that the difference is identically zero. In particular, taking x = m and y = 0, for any m > 1, we 
find 

1(0, m) = mT(m,0) . 

The union of the in-graphs of the sites (0, m), for 1 < m < n, is the set of all sites (x,y) with 
x > and y > that lie under the oriented path of the dual percolation graph H' that starts at 
(—1/2, n + 1/2). Each of these sites lies at the centre of a unit square with vertices (x±l/2, y±l/2), 
and the union of these squares is the region bounded by the dual percolation path and the lines 
x = —1/2 and y = 1/2. Reflecting this region in the line y = x + 1/2 we obtain a sample of the 
region bounded by a simple harmonic urn path and the co-ordinate axes. The expected number of 
unit squares in this region is therefore X^m=i -^(0, m.) , so we are done. □ 

Proof of Theorem 12.101 The argument uses a similar idea to the proof of Theorem 12.151 this 
time for the percolation model on the whole of 1Z. Choose a continuous branch of the argument 
function on 7Z. Let I+(v) denote the expected number of points w with arg(w) > in the in-graph 
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of v in H, including v itself if arg(u) > 0. Arguing as before, if the projection of v to Z 2 is (x,y), 
then I+(v) satisfies the boundary condition I+(v) = for arg(f) < 0, and the recurrence relation 

I+(v) = 1 + r— 7^—1 J +(v + (sgn(y), 0)) + |x| pr, /+(« + (0, - sgn(x))), 

\x + sgn(y)| + \y\ \x\ + \y - sgn(x)| 

where on the right-hand side is evaluated at two of the neighbours of v in the graph G. Setting 
J+(y) := I+(v) / (\x(v)\ + \y(v)\), we have a recurrence relation for J + : 

J+(v) = 1 + |y| J + (u + (sgn(y),0))+ J + (t> + (0,-sgn(x)). 

fI + \y\ fI + 1 2/ 1 fI + 1 2/ 1 

The same recurrence relation and boundary conditions hold for T + (v), where T+(w) is the 
expected time to hit the set arg z > in 1Z in the fast embedding, starting from a vertex w. Here, 
v is the vertex of G at the same distance from the origin as v, satisfying arg(v) = — arg(u). The 
reasoning of the previous proof shows that T+(v) = J+{v) for all vertices v with argt; < tt/2, and the 
argument may be repeated on the subsequent quadrants to show by induction that T + (v) = J+(v) 
for all vertices v. We therefore have the lower bound 



J+(v)=T + (v)> 



aig(v) 
tt/2 



mm n [r f ] 



The asymptotic expression ([7]), together with trivial lower bounds for small n, implies that inf n E n [rj] > 
0. Therefore, as v varies over the set of vertices of G with a given projection (x, y), both J+(v) and 
I+(v) tend to infinity with arg(u). Note that I+(v) is a lower bound for I(v), and I(v) depends 
only on the projection (x,y). It follows that 

E[J(u)] = oo. (60) 

Recall that the oriented path of H' starting at (m + h, — i) explores the outer boundary of the 
in-graph of the set S of vertices with arg(v) = and x < m, and that it can be mapped via $ 
onto a path of the leaky simple harmonic urn. Let A denote the area swept out by this path up 
until time r (the hitting time of {(x,y) : \x\ + \y\ = 1}). The mapping <3? from the vertices of G' to 
Z can be extended by affine interpolation to a locally area-preserving map from n to R 2 \ (0,0). 
So A is equal to the area swept out by the dual percolation path until its projection hits the set 
{(±2, ±2)}- Since the expected number of points in the in-graph that it surrounds is infinite, we 
have E[A] =00. □ 



9.3 Exact formulae for expected traversal time and enclosed area 

In this section we present some explicit, if mysterious, formulae for the expected area enclosed 
by a quadrant-traversal of the urn process and the expected quadrant-traversal time in the fast 
embedding. We obtain these formulae in a similar way to our first proof of Lemma 13.31 and they 
are reminiscent, but more involved than, the formulae for the Eulerian numbers. There is thus 
some hope that the asymptotics of these formulae can be handled as in the proof of Lemma 14.51 
which gives a possible approach to the resolution of Conjecture 12.131 

Lemma 9.3. E n [Area enclosed] and E n [rj] are rational polynomials of degree n evaluated at e: 
E^Area = £ £ („ - i « -*)l V ~ £ M < 61) 

i=l x=l v J y 1 \ fc=0 / 
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n ' jn—x—i—lj\j^ \ s j n ~~ * / - - »t 



»=1 x=l v 2 v ; V fc=0 / 

Proof. The expected area enclosed can be obtained by summing the probabilities that each unit 
square of the first quadrant is enclosed. That is, 

n oo 

E n [Area enclosed] = y]P w ((ff, y) lies on or below the urn path) . 

x=l y=l 

In terms of the slow continuous-time embedding of Section [31 (x,y) lies on or below the urn path 
if and only if Y^Zl Kj < YJi= x Let 

n y—1 

Rn,x,y = ^ ] i^i ~ ^ ] jCj i 
i=x j=l 

so that 

n oo 

E n [Area enclosed] = y P (Rn,x,y > 0) . 
x=l y=l 

The moment generating function of R n ,x,y is 

L PV n,x,y)\ LL 1 _ i QLL 1+ jQ 1 _ ft ^ 1 + jQ ' 

where 

jn-x+y-lj^ i (_;Q(n-j) 



a; 



(i + y — l)!(i — x)!(n — i)! 
Now the density of R n ,x,y at u; > is 

Eexp(t«/i) 
Oii : ; 

I 

l=X 

so that F(R n , x ,y > 0) = Y17=x a i- Therefore 



n oo n 



E n [Area enclosed] = 



(i + y-l)!(i-s)!(n-i)! ' 

x=l y=l i=x 

The series converges absolutely so we can rearrange to obtain (|6ip . By the first equality in Theo- 
rem [2JJll we find that E n [ry] is also a rational polynomial of degree n evaluated at e. After some 
simplification we obtain (|62[) . □ 



A remarkable simplification occurs in the derivation of (I62p from (161 ft . so it is natural to try 
the same step again, obtaining 

i n 1 ;n-x—i-2;\( -i\n-i ( 1-1 ,-fc \ 

iCE. W - E„_ l[T/1 ) . £ £ 1 ^ e< -El) ■ 

i=l x =l ^ I v ' \ fc=o / 

In light of Theorem 12.141 and Conjecture 12.131 we would like to prove that this expression decays 
exponentially as n — > oo. Let us make one more observation that might be relevant to Conjecture 
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12.131 Define F(i) = Yl x =i (i-x)H x > which can be interpreted as the expected number of distinct 
balls drawn if we draw from an urn containing i distinguishable balls, with replacement, stopping 
when we first draw some ball for the second time. We have already seen, in equation (|62p . that 

perhaps one could exploit the resemblance to the formula 

(-1) 

i=l 




n 



E n [l/Z k+1 ] = 
but we were unable to do so. 




10 Other stochastic models related to the simple harmonic urn 
10.1 A stationary model: the simple harmonic flea circus 

In Section [3l we saw that the Markov chain has an infinite invariant measure 7r(n) = n. We 
can understand this in the probabilistic setting by considering the formal sum of infinitely many 
independent copies of the fast embedding. Here is an informal description of the model. At time 

0, populate each vertex of Z 2 with an independent Poisson-distributed number of fleas with mean 

1. Each flea performs a copy of the process (A(t), B(t)), independently of all the other fleas. Let 
Nt(m,n) denote the number of fleas at location (m,n) at time t. 

As we make no further use of this process in this paper, we do not define it more formally. 
Instead we just state the following result and sketch the proof: compare the lemma in [161 §2], 
which the authors attribute to Doob. 

Lemma 10.1. The process {Nt(m,n) : m,n € Z} is stationary. That is, for each fixed time t > 0, 
the array Nt(m,n), m,n G Z consists of independent Poisson(l) random variables. The process is 
reversible in the sense that the ensemble of random variables Nt(m,n),0 < t < c has the same law 
as the ensemble N c —t(m, —n), < t < c, for any c > 0. 

This skew-reversibility allows us to extend the stationary process to all times tel. 

To see that the process has stationary means, note that the expectations E[Nt(m,n)] satisfy a 
system of coupled differential equations: 

^E[N t (m, n)] = —(\m\ + |n|)E[iVt(m, n)] + |m|E[iVt(m, n — sgn(m))] + \n\E[N t (m + sgn(n), n)], 
the solution to which is simply E[N t (m, n)] = 1 for all t, m and n. 

To establish the independence of the variables N t (a,b) when t > is fixed, we use a Poisson 
thinning argument. That is, we construct each variable No(m, n) as an infinite sum of independent 
Poisson random variables N(m, n, a, b) with means 

E[N{m, n, a, b)] = F((A(t),B(t)) = (a, b) \ (A(fl),B(0)) = (m, n)). 

The variable N(m, n, a, b) gives the number of fleas that start at (m, n) at time and are at (a, b) at 
time t. Then Nt(a, b) is also a sum of infinitely many independent Poisson random variables, whose 
means sum to 1, so it is a Poisson random variable with mean 1. Moreover, for (a, b) ^ (a', b'), the 
corresponding sets of summands are disjoint, so N t (a,b) and N t (a',b') are independent. 
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10.2 The Poisson earthquakes model 

We saw how the percolation model of Section 12.41 gives a static grand coupling of many instances 
of (paths of) the simple harmonic urn. In this section we describe a model, based on 'earthquakes', 
that gives a dynamic grand coupling of many instances of simple harmonic urn processes with 
particularly interesting geometrical properties. 

The earthquakes model is defined as a continuous-time Markov chain taking values in the group 
of area-preserving homeomorphisms of the plane, which we will write as 

& t : m 2 -> m 2 , teR. 

It will have the properties 

• ©o is the identity, 

• 6 t (0,0) = (0,0), 

• &t acts on Z 2 as a permutation, 

• & s o has the same distribution as G s _j, and 

• for each pair (xo,yo) 7^ £ <^ 2 > the displacement vector 

&t(xi,Vi) ~ &t(x ,y ) 

has the distribution of the continuous-time fast embedding of the simple harmonic urn, start- 
ing at (xi - x ,yi - y ). 

In order to construct @t, we associate a unit-rate Poisson process to each horizontal strip 
H n := {(x,y) E ]R 2 : n < y < n+1}, and to each vertical strip V n := {(x,y) 6R 2 : n < x < n+1}, 
(where n ranges over Z). All these Poisson processes should be independent. Each Poisson process 
determines the sequence of times at which an earthquake occurs along the corresponding strip. An 
earthquake is a homeomorphism of the plane that translates one of the complementary half-planes 
of the given strip through a unit distance parallel to the strip, fixes the other complementary half- 
plane, and shears the strip in between them. The fixed half-plane is always the one containing the 
origin, and the other half-plane always moves in the anticlockwise direction relative to the origin. 

Consider a point (xo>3/o) S M 2 . We wish to define &t(xo,Vo) for all t > 0. We will define 
inductively a sequence of stopping times e$, and points (xi,?/j) G M 2 , for i £ Z + . First, set £o = 0. 
For i£N, suppose we have defined Vi-x) and £1-1. Let be the least point greater than £j_i 

in the union of the Poisson processes associated to those strips for which (xj_i,yj_i) and (0,0) do 
not both lie in one or other complementary half-plane. This is a.s. well-defined since there are only 
finitely many such strips, and a.s. there is only one strip for which an earthquake occurs at time 
£j. That earthquake moves (xj_i,yj_i) to (xi,yi). Note that £j — £j_i is an exponential random 
variable with mean + [~|?/j_i|]), conditionally independent of all previous jumps, given 

this mean. Since each earthquake increases the distance between any two points by at most 1, it 
follows that a.s. the process does not explode in finite time. That is, £j — > oo as % — > oo. Define 
3/0 ) to be (xi,yi), where £j <t < £j+i. The construction of &t for t < is similar, using the 
inverses of the earthquakes. 
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Note that we cannot simply define &t for t > to be the composition of all the earthquakes 
that occur between times and t, because almost surely infinitely many earthquakes occur during 
this time; however any bounded subset of the plane will only be affected by finitely many of these, 
so the composition makes sense locally. 

The properties listed above follow directly from the construction. For (xq, yo), (xi, y±) E Z 2 , 
the displacement vector (Axt,Ay t ) = &t(xi,yi) — <5t(xo,yo) only changes when an earthquake 
occurs along a strip that separates the two endpoints; the waiting time after t for this to occur is 
exponentially distributed with mean l/(|Ax(| + |Ayt|), and conditionally independent of &t given 
(Ax t ,Ay t ). 




Figure 3: A simulation of &t, shown at times t = 0, vr/4, tt/2, 37r/4 acting on a 20 x 20 box. 
The model is spatially homogeneous in the following sense. Fix some (o, b) G Z 2 and define 

&t(x, y) = &t(x + a,y + b)- & t (a, b) . 
Then &t has the same distribution as <5f 

Lemma 10.2. Define an oriented polygon T by the cyclic sequence of vertices 

((xi,2/i), • • • , (x n _i,y n _i), (x n ,y n ), (xi,yi)) , {xi,yi) G I? . 
The signed area enclosed by the polygon Ft, given by 

(&t{x 1 ,y 1 ),6 t {x 2 ,y2),-- ■ , &t(x n ,y n ), &t(xi, yi)) , 

is a martingale. 
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Proof. For convenience we write (xi(t) , y%{t)) = &t((xi, yi)). The area enclosed by the oriented 
polygon Tt is given by the integral % Jr X( ^V ~ 2/dx, which we can write as 

1 n 

- (xi(t)y i+ i(t) - x i+1 {t)yi{t)) , 
i=l 

where (x n +l> y n +i) is taken to mean (x\,yi). So it suffices to show that each term in this sum 
is itself a martingale; let us concentrate on the term xi(t)y2(t) — X2(t)y\(t), considering the first 
positive time at which either of (xi(t), yi(t)) or (x2(t), 2/2W) jumps. There appear to be at least 36 
cases to consider, depending on the ordering of {0, x±,X2} and {0,2/1,2/2}, but we can reduce this 
to four by taking advantage of the spatial homogeneity of the earthquakes model, described above. 
By choosing (a, b) suitably, and replacing (3 by & , we can assume that Xj, yi > 0, for i = 1, . . . , n. 
Furthermore, swapping the indices 1 and 2 only changes the sign of x\{t)y2(t) — X2(t)yi(t), so we 
may also assume that x\ < X2- Suppose that the first earthquake of interest is along a vertical 
line. Then with probability X\jx2 it increments both y\ and 2/2 an d otherwise it increments only 
2/2- The expected jump in xi{t)y2{t) — X2(t)yi(t) conditional on the first relevant earthquake being 
parallel to the y-axis is therefore 

X\ X2 — X\ 

— ((2:1(2/2 + 1) - 2:2(2/1 + 1)) - (2:12/2 - X2yi))-\ ((2:1(2/2 + 1) - 2:22/1) - (2:12/2 - 2:22/1)) = . 

X 2 X 2 

A similar argument shows that the expected jump in X\(t)y2{t) — 2:2(^)2/1 (£) conditional on the first 
relevant earthquake being parallel to the x-axis is also zero. □ 



10.3 Random walks across the positive quadrant 

In this section we describe another possible generalization of the simple harmonic urn that has 
some independent interest. We define a discrete-time process (A n , -Bn)nez+ on IR 2 based on the 
distribution of an underlying non-negative, non-arithmetic random variable X with ~E[X] =/i E 
(0,oo) and VarLY] = a 2 G (0, 00). Let Xi,X2,... and X[,X 2 ,... be independent copies of X. 
Roughly speaking, the walk starts on the horizontal axis and takes jumps (—X^Xi) until its first 
component is negative. At this point suppose the walk is at (— r, s). Then the walk starts again at 
(s,0) and the process repeats. We will see (Lemma I10.4|) that in the case when X ~ U(0, 1), this 
process is closely related to the simple harmonic urn and is consequently transient. It is natural to 
study the same question for general distributions X. It turns out that the recurrence classification 
depends only on fi and a 2 . Our proof uses renewal theory. 

We now formally define the model. With X, X n , X' n as above, we suppose that E[X 4 ] < 00. Let 
(Aq,Bq) = (a, 0), for a > 0. Define the random process for n £ Z + by 



(An - X' n , B n + X n ) \iA n >0 
(B n ,0) if^„<0. 



(Ai+lj-Bn+l) 

Theorem 10.3. Suppose ELY 4 ] < 00. The walk (A n ,B n ) is transient if and only if fi 2 > o 2 . 



Set ro := — 1 and for fceN, 

T k := min{n > r fc _i : A n < 0}. 

Define := — (t^-i + 1). That is, T^ is the number of steps that the random walk takes to 
cross the positive quadrant for the kth time. 
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Lemma 10.4. If X ~ ^(0, 1) and the initial value a is distributed as the sum of n independent 
U(0, 1) random variables, independent of the Xi and X[, then the distribution of the process (Tk)keN 
coincides with that of the embedded simple harmonic urn process (Z^ke'N conditional on Zq = n. 



Proof. It suffices to show that T\ = t\ has the distribution of Z\ conditional on Zq = n and that 
conditional on t\ the new starting point ^4i+ Tl , which is B Tl , has the distribution of the sum of r\ 
independent U (0, 1) random variables. Then the lemma will follow since the two processes (r^, B Tk ) 
and (Z k ) are both Markov. To achieve this, we couple the process (A n , -B n ) up to time T\ with the 
renewal process described in Section [H To begin, identify a with the sum (1 — xi) + • • • + (1 — Xn)- 
Then for k G {1, . . . ,N(n) — n}, where N(n) > n is as defined at (|15p . we identify X' k with Xn+k- 
For m < Ti we have 

m n+m 

A m = a-J2 x 'i = n-^Xi, 

i=l i=l 

so in particular we have AiV(n)— n— l — ^ anc ^ ^N(n)-n < by definition of N(n). Hence T\ = N(n)—n 
has the distribution of Z\ by Lemma l4.1i Moreover, j4i+ t1 = B T1 is the sum of the independent 
U(0, 1) random variables Xi, i = 1, . . . , t\. □ 

Thus by Theorem 12.11 in the case where X is U(0, 1), the process (A n ,B n ) is transient, which 
is consistent with Theorem 110.31 since in the uniform case \i = 1/2 and a 2 = 1/12. To study the 
general case, it is helpful to rewrite the definition of {A n , B n ) explicitly in the language of renewal 
theory. Let So = S' = and for n G N set S n = X^=i-^i = Y17=i^'i- D° nne the renewal 
counting function for S' n for a > as 

N(a) := min{n G Z + : S' n > a} = 1 + max{n G Z + : S' n < a}. 

Then starting at (Aq,Bq) = (a, 0), a > 0, we see T\ = N(a) so that B Tl = S^^ a y To study the 
recurrence and transience of (A n ,B n ) it thus suffices to study the process (R n )nez + with Rq := a 
and R n having the distribution of SV^.) given R n -\ = x. The increment of the process R n starting 
from x thus is distributed as A(x) := Sj^m —x. It is this random quantity that we need to analyse. 

Lemma 10.5. Suppose that E[X 4 ] < oo. Then as x — >■ oo, E[|A(x)| 4 ] = 0(x 2 ) and 

2 2 

E[A(x)] = + 0(x~ l ) 

E[A(x) 2 ] = — + 0(1). 
A 4 

Proof. We make use of results on higher-order renewal theory expansions due to Smith |39| (note 
that in |39j the renewal at is not counted). Conditioning on N(x) and using the independence of 
the Xi, X[, we obtain the Wald equations: 

E[S N{x) ] = (M[N(x)]\ Var^j] = a 2 E[iV(x)] + /i 2 Var[iV(x)]. 

Assuming E[X 3 ] < oo, [391 Thm. 1] shows that 

m(*)] = - + ^r- + o(x- 1 ), 

X(T 

Var[JV(x)] = — + 0(1). 
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The expressions in the lemma for E[A(x)] and E[A(x) 2 ] follow. 
It remains to prove the bound for E[|A(x)| 4 ]. Write A(x) as 

S N{X) -x = (S N(x) - tiN(x)) + (jiN(x) - fjE[N{x)]) + (jJ®.[N(x)] - x). (63) 

Assuming E[X 2 ] < oo, a result of Smith \39\ Thm. 4] implies that the final bracket on the right- 
hand side of (j63j) is 0(1). For the first bracket on the right-hand side of ()63|) . it follows from the 
Marcinkiewicz-Zygmund inequalities |14( Cor. 8.2, p. 151] that 

E[(S N{x) -fiN(x)) 4 ]<CE[N(x) 2 ], 

provided ELY 4 ] < oo. This last upper bound is 0(x 2 ) by the computations in the first part of this 
proof. It remains to deal with the second bracket on the right-hand side of ()63|) . By the algebra 
relating central moments to cumulants, we have 

E[(jiN(x) - LiE[N(x)]) A ] = fi A (k 4 (x) + 3k 2 (x) 2 ), 

where k r (x) denotes the rth cumulant of N(x). Again appealing to a result of Smith [39^ Cor. 2, 
p. 19], we have that £2(2;) and k±{x) are both O(x) assuming ELY 4 ] < 00. (The fact that [39] does 
not count the renewal at is unimportant here, since the rth cumulant of N(x) =t 1 differs from 
k r (x) by a constant depending only on r.) Putting these bounds together, we obtain from (j63j) and 
Minkowski's inequality that E[(S N ( X ) - x) 4 ] = <3(x 2 ). □ 

To prove Theorem 110.31 we basically need to compare E[A(x)] to E[A(x) 2 ]. As in our analysis 

1/2 

of Zk, it is most convenient to work on the square-root scale. Set V n := Rn ■ 
Lemma 10.6. Suppose that E[X 4 ] < 00. Then there exists 5 > such that as y — >■ 00, 

E[K+1 -V n \V n 

E[(V n+1 - V n f I V n 
E[|K+i - K| 3 I V n 

Proof. The proof is similar to the proof of Lemma 17.71 except that here we must work a little 
harder as we have weaker tail bounds on A(x). Even so, the calculations will be familiar, so we 
do not give all the details. Write E x [-] for E[- | R n = x] and similarly for P x . From Markov's 
inequality and the 4th moment bound in Lemma llU.5t we have for e G (0, 1) that 

P x (\A(x)\ > x 1 ^) = 0(x 4£ ~ 2 ). (64) 

We have that for x > 0, 

E[K+i -Vn\V n = x 1 ' 2 ] = E x [R n ^ - R l J 2 ] = E x [(x + A(x)) 1 / 2 _ 
Here we can write 

(x + A(x)) 1 / 2 - x 1 / 2 = [(x + A(x)) 1 / 2 - x 1 / 2 ]l{|A(x)| < x 1 - 6 } + R x + R 2 , (65) 

for remainder terms R±, R 2 that we define shortly. The main term on the right-hand side admits a 
Taylor expansion and analysis (whose details we omit) in a similar manner to the proof of Lemma 



y] 



y] 



2\21 



m(y 2 )} E[A(y 2 ) 
E[A(y 2 ) 2 ; 



8y 3 
+ 0(y~ 5 ) 



+ 0(y 



-1— <S\ 



y\ = 0(1) 
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17.71 and contributes to the main terms in the statement of the present lemma. The remainder 
terms in (I65D are 

Hi = [Or + A(x)) 1 / 2 - x*-' 2 ]l{A(x) > x 1 ^}, R 2 = [(x + A(x)) 1 / 2 - x 1 / 2 ]l{A(x) < -x 1 " 6 }. 

For the second of these, we have \R 2 \ < x 1/2 l{A(x) < —x e }, from which we obtain, for r < 4, 
J&x[\R2\ r ] = 0(x 4£+ ( r - 4 )/ 2 ), by dMD- Taking e small enough, this term contributes only to the 
negligible terms in our final expressions. For R±, we have the bound 

|J?i| < C(l + \A{x)\)W +e l{A(x) > x 1 " 5 } 

for some C £ (0, oo) not depending on x, again for e small enough. An application of Holder's 
inequality and the bound ([641 implies that, for r < 4, for any e > 0, 



®x[\Ri\ r ] < C(E x [(l + |A(x)|) 4 ]) lIi T M (P :i; (A(x) > x 1 ^)) 1 "^ = O^+C'- 4 )/ 2 ). 

It is now routine to complete the proof. □ 
Proof of Theorem 110.31 For the recurrence classification, the crucial quantity is 



2yE[V n+1 -V n \V n = y}- E[(V n+1 - V n f \V n = y] = E[A(y z )] - ~ L ~ y » ' J + 0(y- d ), 



E[Ajy_ 2 ^ 

2y 2 



by Lemma 110.61 Now by Lemma 110.51 this last expression is seen to be equal to 

Now |26[ Thm. 3.2] completes the proof. □ 

Remarks, (i) To have some examples, note that if X is exponential, the process is recurrent, while 
if X is the sum of two independent exponentials, it is transient. We saw that if X is U{0, 1) the 
process is transient; if X is the square-root of a U(0, 1) random variable, it is recurrent. 

(ii) Another special case of the model that has some interesting features is the case where X is 
exponential with mean 1. In this particular case, a calculation shows that the distribution of T^+i 
given Tfc = m is negative binomial {m + 1, 1/2), i.e., 

P(T fe+1 = j | T k = m )=( J + m ] 2—i-i, (j e Z+ ). 

Since /i 2 = a 2 , this case is in some sense critical, a fact supported by the following branching 
process interpretation. 

Consider a version of the gambler's ruin problem. The gambler begins with an initial stake, a 
pile of mo chips. A sequence of independent tosses of a fair coin is made; when the coin comes up 
heads, a chip is removed from the gambler's pile, but when it comes up tails, a chip is added to a 
second pile by the casino. The game ends when the gambler's original pile of chips is exhausted; 
at this point the gambler receives the second pile of chips as his prize. The total number of chips 
in play is a martingale; by the optional stopping theorem, the expectation of the prize equals the 
initial stake. As a loss leader, the casino announces that it will add one extra chip to each gambler's 
initial stake, so that the game is now in favour of the gambler. Suppose a gambler decides to play 
this game repeatedly, each time investing his prize as the initial stake of the next game. If the 
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casino were to allow a zero stake (which of course it does not), then the sequence of augmented 
stakes would form an irreducible Markov chain Sk on N. Conditional on Sk = m, the distribution 
of Sk+i is negative binomial (m + 1/2,1/2). So by the above results, this chain is recurrent. It 
follows that with probability one the gambler will eventually lose everything. 

We can interpret the sequence of prizes as a Galton-Watson process in which each generation 
corresponds to one game, and individuals in the population correspond to chips in the gambler's pile 
at the start of the game. Each individual has a Geo(l/2) number of offspring (i.e., the distribution 
that puts mass 2 _1 ~ fe on each k £ Z+), being the chips that are added to the prize pile while that 
individual is on top of the gambler's pile, and at each generation there is additionally a Geo(l/2) 
immigration, corresponding to the chips added to the prize pile while the casino's bonus chip is on 
top of the gambler's pile. This is a critical case of the Galton-Watson process with immigration. 
By a result of Zubkov if we start at time with population 0, the time r of the next visit to 
has pgf 

Since this tends to 1 as s /* 1, we have P(r < oo) = 1. In fact this can be deduced in an elementary 
way as follows. The pgf of the Geo(l/2) distribution is f(s) = 1/(2 — s), and its n th iterate, the 
pgf of the n th generation starting from one individual, is f(s) = (n — (n — l)s)/((n + 1) — ns). In 
particular the probability that an individual has no descendants at the n th generation is n/(n + 1). 
If So = 1, then Sk = 1 if and only if for each j = 0, . . . , k — 1 the bonus chip from game j has no 
descendants at the (k — j) th generation. These events are independent, so 

P(St=1 | So=1)= n<w = *TT, 

3=0 J 

which sums to oo over k £ N so that the Markov chain is recurrent (see e.g. [H Prop. 1.2, §1]). 

The results of Pakcs [33j on the critical Galton-Watson process with immigration show that 
the casino should certainly not add two bonus chips to each stake, for then the process becomes 
transient, and gambler's ruin will no longer apply. 
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